Automated storage and retrieval device and method

ABSTRACT

A device and method for the automated storage and retrieval of trays holding subject matter. A computer system is programmed to control a storage gantry to move the trays between a storage rack and an automated machine. In a preferred embodiment, the subject matter in the trays is a plurality of micro-well plates in which microscopic crystals may be growing and the automated machine is configured to inspect and classify microscopic crystals. The automated machine has an indexing device for sequentially placing microscopic crystals in camera-view of a camera and a control computer is programmed to control the indexing device and to cause the camera to take images of the microscopic crystals and then transfer the images to a classifying processor where the images are classified. In a preferred embodiment, the microscopic crystals are protein crystals that have been grown in the wells of micro-well plates.

The present invention relates to automated storage and retrieval devicesand methods for using, and in particular to such devices and methodsused in conjunction with inspection devices for sequentially inspectingmicroscopic crystals. This application is a continuation-in-part of Ser.No. 10/627,386 filed Jul. 25, 2003, and is a continuation-in-part ofSer. No. 10/057,226, filed Jan. 25, 2002 and soon to issue as U.S. Pat.No. 6,637,473, which is a continuation in part of Ser. No. 09/702,164filed Oct. 30, 2000 now U.S. Pat. No. 6,360,792 and Ser. No. 09/982,048filed Oct. 18, 2001 all of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

The determination of the three dimensional atomic structure of matter isone of the most important areas of pure and applied research. One way inwhich the three dimensional atomic structure of matter can be determinedis through X-ray crystallography. X-ray crystallography utilizes thediffraction of X-rays from crystals in order to determine the precisearrangement of atoms within the crystal. The result may reveal theatomic structure of substances such as metal alloys, deoxyribonucleicacid (DNA), or the structure of proteins.

There are very important benefits to knowing the accurate molecularstructure of a protein crystal. For example, once the molecularstructure is known, a drug designer can more effectively developeffective therapeutic agents and drugs. However, despite its promises,X-ray crystallography is limited by the fact that it is very difficultto grow successful crystals.

Prior Art Method of Growing Crystals

Protein crystals are commonly grown in the wells of micro-well plates. Amicro-well plate is also known as a micro-titer plate or a microplate.Micro-well plates typically come with either 24, 48, 96, 384 or 1536wells. A 96-well micro-well plate is shown in detail in FIG. 2. Thereare a variety of methods in which protein crystals may be grown. Fivecommon ways are summarized below.

Hanging Drop Method

One of the main techniques available for growing crystals, known as thehanging-drop or vapor diffusion method, is a method wherein a drop of asolution containing protein is applied to a glass cover slip and placedupside down in an apparatus such as a vapor diffusion chamber whereconditions lead to supersaturation in the protein drop and theinitiation of precipitation of the protein crystal.

Sitting Drop Method

Another method is the sitting drop method where the drop sits in a smallwell adjacent the growing solution instead of hanging over it. Thismethod provides a more stable drop and location.

Aqueous Drop in Oil Method

Another method is the aqueous drop in oil method. The drop is placed ina micro-well and is covered with an oil based solution. The drop staysat the bottom of the well as the crystal grows.

Dialysis Method

In another method referred to as the dialysis method (also calledmicrobatch crystallization), the protein solution is contained within asemi-permeable size exclusion membrane and then placed in a solution offixed pH and precipitant concentration. As the precipitant diffusesthrough the membrane into the protein compartment, the solubility of theprotein is reduced and crystals may form.

Gel Crystal Growth Method

This method involves the placement of a gel into the end of smalldiameter glass capillaries. After the solutions have gelled, a proteinsolution is placed into one end (top) of the capillary and the other endis submerged in a solution of precipitating agent. If the conditions areappropriately selected, crystal growth occurs at a point in the gelwhere the protein and precipitating agent reach the properconcentrations as the solutions slowly mix by diffusion. Since this is adiffusion limited process, it thus only occurs after an extended periodof time. Crystals however, grown by this method are often larger and ofhigher quality.

Regardless of the method chosen, protein crystal growth is a verydelicate and time-consuming process. It can take several days to severalmonths before crystals of sufficient size and quality are grown andready for x-ray crystallography. The current minimum size that istypically stated is a crystal of at least 50 microns thick by 100microns in extent. The protein crystal growing environmental conditionsneed to be rigorously maintained, from the chemistry, to the surroundingair humidity and temperature, cleanliness to prevent contamination, andeven lighting conditions. A protein crystallographer working withunknown protein families may only be about 5% successful in growingproper sized quality crystals. With this success rate, for example, a96-well micro-well plate may only have 5 wells in which good crystalsare growing.

Prior Art Inspection of Crystal Growth

Currently, a laboratory technician, or operator, aided by a microscopeand a laboratory notebook manually inspects crystals grown in micro-wellplates. To inspect a micro-well plate, a laboratory technician dons aclean-room gown suit and enters a cold room in which the crystals aregrowing. The technician then puts a micro-well plate underneath themicroscope and examines each well in the micro-well plate until all ofthe wells in the micro-well plate have been inspected. The technicianthen makes a mental judgement as to how he shall classify (also known as“score”) the crystal. For example, the technician may feel that he isobserving an image that shows “grainy precipitation” or “uglyprecipitation”. Or, he may feel that the image shows “no crystalgrowth”. The technician then records the classification into alaboratory notebook.

The above system is riddled with opportunities for human error. Anoperator, manually inspecting a 96-well micro-well plate will takeapproximately 5 to 20 minutes depending on the skill of the operator andthe number of wells that contain interesting features, microcrystals, orcrystals. The operator may be subject to physical fatigue, suffereyestrain, and may be uncomfortably cold in the temperature controlledand generally high humidity room. The operator can be tired and confusedand can easily make errors in manually recording data in the notebook.For example, the operator may observe crystal growth at well H5 (FIG.2), but incorrectly record in the notebook that the crystal growth wasat well H6. Additional transcription errors may occur when the data istransferred to a computer database.

Research efforts are underway to try to solve the above problem, butthey are inadequate for the needs of the industry. One such effort isdescribed in Jurisica et al. “Intelligent Decision Support for ProteinCrystal Growth” IBM systems Journal, Vol. 40, No 2, 2001. Another sucheffort is described at the Website www.dsitech.com.

Current Problems with Micro-Well Plate Storage and Retrieval Procedures

Typically, after a technician has inspected a micro-well plate forcrystal growth, the micro-well plate is stored until it is time toinspect it again. The growing of protein crystals in micro-well platesand the accompanying inspection of the micro-well plates for successfulcrystal growth are procedures that are typically carried outconcurrently in large quantities in laboratories. For example, a typicallab at any given moment may have literally thousands of micro-wellplates in which protein crystals are attempting to grow. The growthcycle of a protein crystal can be approximately 6 months. During the 6month time period, a micro-well plate may be inspected up toapproximately 12 times. If there are thousands of micro-well plates thatrequire inspection, it can be a very time consuming task to manuallymove the micro-well plate from its storage location, place it under amicroscope, record the results, and then move it back to its appropriatestorage location. Moreover, there is tremendous opportunity for atechnician to forget where a particular micro-well plate belongs. Or, atechnician handling such a large quantity of micro-well plates caneasily drop or otherwise damage the micro-well plates he is handling.

What is needed is a better device and method for storing and retrievingtrays containing micro-well plates.

SUMMARY OF THE INVENTION

The present invention provides a device and method for the automatedstorage and retrieval of trays holding subject matter. A computer systemis programmed to control a storage gantry to move the trays between astorage rack and an automated machine. In a preferred embodiment, thesubject matter in the trays is a plurality of micro-well plates in whichmicroscopic crystals may be growing and the automated machine isconfigured to inspect and classify microscopic crystals. The automatedmachine has an indexing device for sequentially placing microscopiccrystals in camera-view of a camera and a control computer is programmedto control the indexing device and to cause the camera to take images ofthe microscopic crystals and then transfer the images to a classifyingprocessor where the images are classified. In a preferred embodiment,the microscopic crystals are protein crystals that have been grown inthe wells of micro-well plates.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a preferred embodiment of the present invention.

FIG. 2 shows a micro-well plate.

FIG. 3 shows a top view of the fixture plate.

FIGS. 4 and 5 show top views of micro-well plates on the fixture plate.

FIG. 6 shows a block diagram of a preferred embodiment of the presentinvention.

FIG. 7 shows a preferred monitor.

FIGS. 8-10 and 18-25 show steps in the sequence of operations of apreferred embodiment of the present invention.

FIG. 11 shows hanging drops of liquid in a micro-well plate.

FIG. 12 shows an example of aqueous drop in oil protein crystallization.

FIG. 13 shows a top view of a micro-well plate on the fixture plate.

FIG. 14 shows a side view of the light source shining upwards onto amicro-well plate.

FIG. 15 shows a magnified view of two wells of a micro-well plate,wherein each well has a drop of liquid.

FIGS. 16 and 17 show a detail view of the drops of liquid shown in FIG.15.

FIG. 26 shows a preferred monitor screen after a run has been completed.

FIGS. 27 and 28 show details of other preferred monitor screens.

FIG. 29 shows a hanging drop of liquid with crystal growth.

FIG. 30 shows a preferred embodiment of the present invention.

FIG. 31 shows a flowchart of an auto-focus subroutine of the presentinvention.

FIG. 32. shows a flowchart of a focus value subroutine.

FIG. 33 shows a flowchart of the auto score and classify subroutine.

FIGS. 34 a-34 d show flowcharts of the classify subroutine.

FIGS. 35 a-35 b show the sub-classification of the crystal class.

FIG. 36 shows the main program flow.

FIG. 37 illustrates a side view illustrating dual filters in the lightpath.

FIG. 38 illustrates a top view of the drive mechanism for the rotatablelinear polarized filter.

FIG. 39 illustrates a top view of a second filter wheel.

FIG. 40 shows the connectivity of another preferred embodiment.

FIGS. 41A and 41B shows another preferred embodiment of the presentinvention.

FIG. 42 shows the top view of a preferred tray.

FIG. 43 shows a preferred work cell area.

FIG. 44 shows a block diagram of a preferred embodiment of the presentinvention.

FIGS. 45-77 show a sequence of operation of a preferred embodiment ofthe present invention.

FIG. 78 shows a perspective view of a preferred tray.

FIG. 79 shows another view of the tray of FIG. 78.

FIG. 80 shows another view of the tray of FIG. 78.

FIG. 81 shows another view of the tray of FIG. 78.

FIG. 82 shows another view of the tray of FIG. 78.

FIG. 83 shows another preferred embodiment of the present invention.

FIG. 84 shows a simplified perspective view of a preferred proteomiccrystal verification and inspection system.

FIG. 85 a shows a detailed perspective view of a preferred proteomiccrystal verification and inspection system.

FIG. 85 b shows a detailed perspective view of a preferred indexer.

FIGS. 86-90 illustrate a preferred sequence of operations of thepreferred proteomic crystal verification and inspection system shown inFIGS. 83-85 b.

FIGS. 91-94 show a preferred stand by station.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT

A detailed description of a preferred embodiment of the presentinvention can be described by reference to the drawings.

FIG. 1 shows a preferred embodiment of the present invention. Micro-wellplates 125A-125F are placed on fixture plate 129. In a preferredembodiment, each micro-well plate has 96 wells. Each well has a drop ofliquid in which microscopic protein crystals may be growing. Computer105 automatically controls linear actuators 115, 150 and 160. Linearactuator 115 moves fixture plate 129 along the x-axis. Linear actuator150 moves moving base 154 along the y-axis, and linear actuator 160moves moving plate 162 along the z-axis. n The computer coordinates themovement of the linear actuators to properly position cameras 155 and135 above each well of each micro-well plate 125A-125F in sequence.Preferably, cameras 155 and 135 are high-resolution ⅔ inch CCD cameras,with 1,300 horizontal pixel elements by 1,030 vertical elements. Eachelement is preferably 6.7 microns square with a sensing area of 8.7 mmby 6.9 mm. Cameras 155 and 135 take images of each well and transmit theimages to computer 105 where they are digitized into a image data arrayof approximately 1,296 horizontal pixels by 1,000 vertical pixels by 8bits of gray level representing the intensity (0 to 255) of each pixel.These digitized images are automatically recorded in the database ofcomputer 105 and can be analyzed and scored by an operator via monitor620. The digitized images may be further processed, analyzed, and thecontents of the individual well scored by computer 105 executing programinstructions to perform calculations on the image data array. In apreferred embodiment, computer 105 is connected via acommunication/control line to a computer network. In this manner, thepresent invention can be controlled from a remote computer. Likewise,images and data can be transmitted to the remote computer.

Sequence of Operation of a Preferred Embodiment Micro-well Plates Loadedonto the System

As shown in FIG. 1, six 96-well micro-well plates 125A-125F have beenplaced (either by an operator or by an external loading robot) ontofixture plate 129. A 96 well micro-well plate 125A is shown in FIG. 2.Micro-well plate 125A has wells labeled A1 through H12 and a bar codelabel 220. Micro-well plate 125A is available from Nalge-NuncInternational, with U.S. offices in Rochester, N.Y. Fixture plate 129will hold 24, 48, 96, 384, or 1536 well micro-well plates since themicro-well plate external width and length are fairly standard in theindustry. The 96 well micro-well plate will be used to illustrate thepresent invention.

FIG. 3 shows a top view of fixture plate 129 just prior to loadingmicro-well plates 125A-125F. Fixture plate 129 has six cutout sections131 that are just slightly smaller in length and width than micro-wellplates 125A-125F.

FIG. 4 shows a top view of micro-well plates 125A-125F immediately afterthey have been placed on fixture plate 129.

After the operator has placed micro-well plates 125A-125F onto fixtureplate 129 as shown in FIG. 4, he enters the command into computer 105(FIG. 1 and FIG. 6) to expand detents 510 and 520 (FIG. 5). Theexpansion of detents 510 and 520 firmly secures micro-well plates125A-125F against plate stops 530.

Recording the Bar Code Information for the Micro-well Plates

Computer 105 (FIG. 1) has been programmed to accept inputs from anoperator. FIG. 7 shows a display representing micro-well plates125A-125F on the screen of monitor 620. In FIG. 7, the operator hasmouse clicked on bars 622A-622F, which has caused them to turn green. Byclicking on bars 622A-622F, the operator has selected correspondingmicro-well plates 125A-125F to “run”. The operator sends the command torun the selected micro-well plates by clicking on run bar 623.

In FIG. 8, the operator has given the command to run the selectedmicro-well plates. Micro-well plate 125A has been moved to a positionunderneath cameras 155 and 135. By the utilization of plate sensortransmitter/receiver 186 and reflector 188, information is sent tocomputer 105 reporting that micro-well plate 125A is in positionunderneath the cameras. Plate sensor transmitter/receiver 186 is fixedto support 189 and is aligned to sense whenever a micro-well platebreaks a beam of light emitted by transmitter/receiver 186 and reflectedby a reflector 188. Reflector 188 is mounted on support 191 on theopposite side of the linear actuator 115. The plate sensortransmitter/receiver 186 and reflector 188 are preferably model #E3T-SR13 available from Western Switch Controls of Santa Ana, Calif.

Bar code reader 175 is also mounted to support 189 and is positioned toview bar-code identity label 220 (FIG. 2) attached to micro-well plate125A when it is positioned underneath cameras 155 and 135. Bar-codereader 175 is preferably model # BL601 available from KeyenceCorporation of America of Newark, N.J. Bar-code reader 175 communicateswith computer 105 via a communication line. Information encoded intolabel 220 preferably includes: the plate serial number, the plate type(i.e., 24-well, 48-well, 96-well, 384-well, or 1536-well micro-wellplate), and the well type (i.e., square, or rounded, hanging drop,sitting drop, constrained sitting drop).

The information from plate sensor transmitter/receiver 186 and bar-codereader 175 is transmitted to computer 105 and stored for later useduring the camera inspection and information acquisition phase.

In FIG. 9, linear actuator 115 has moved fixture plate 129 so thatmicro-well plate 125B is underneath cameras 155 and 135. In a fashionsimilar to that described above with regards to micro-well plate 125A,information is transmitted from plate sensor transmitter/receiver 186and bar-code reader 175 to computer 105 and stored for later use duringthe camera inspection and information acquisition phase.

The above described sequence continues until all micro-well plates125A-125F have been sensed and recorded by plate sensortransmitter/receiver 186 and bar-code reader 175.

Then, as shown in FIG. 10, linear actuator 115 moves micro-well plate125A so that it is underneath lens 165 of camera 155. Motor 130 oflinear actuator 160 moves moving plate 162 upward and/or downward asnecessary to properly focus lens 165 on the drop of hanging liquid overwell A1. Preferably, lens 165 is set at a predetermined zoom.

Inspection of Crystals Determining the Position of the Drop of Liquidwithin Each Well

An operation to inspect each well to determine the position of eachhanging drop of liquid is performed on micro-well plate 125A after ithas been moved to the position shown in FIG. 10.

FIG. 11 shows a cross section side view of wells A1-E1 of micro-wellplate 125A. In a preferred embodiment, an attempt has been made to growprotein crystals in the hanging drops in each of the wells of micro-wellplate 125A. FIG. 11 shows hanging drops α1, β1, χ1, δ1, and ε1.

The preferred method for protein crystal growth is the hanging dropmethod. The hanging drop method (also known as vapor diffusion) isprobably the most common method of protein crystal growth. As explainedin the background section, a drop of protein solution is suspended overa reservoir containing buffer and precipitant. Water diffuses from thedrop to the solution leaving the drop with optimal crystal growthconditions.

In FIG. 10, lens 165 of camera 155 is over well A1 of micro-well plate125A (FIG. 2, FIG. 11, and FIG. 13). FIG. 13 shows a top view ofmicro-well plate 125A positioned on fixture plate 129.

FIG. 14 shows a side view of micro-well plate 125A positioned on fixtureplate 129. Support 191 with embedded light source 194 is positioned tothe side of fixture plate 129. Light from light guide 195 is directedupward through cutout 131 (also shown in FIG. 3). Light guide 195 ispositioned between fixture plate 129 and plate 127 such that both platescan move around the light guide 195 without interference. As explainedabove, fixture plate 129 has cutouts 131 (FIG. 3) that are smaller thanthe micro-well plates 125 and located under each well plate, such thatlight from light guide 195 can be projected through the well plates whenthey are brought into position for inspection. In the preferredembodiment, light source 194 is model # A08925 fiber-optic backlightavailable from Aegis Electronics of Carlsbad, Calif.

Camera 155 (FIG. 10) inspects well A1 and transmits an image to computer105 for digitization. As described above, camera 155 preferably (FIG.10) inspects well A1 at a 1× magnification so that every 6.7 micronsquare pixel represents approximately 6.7 square microns on the objectbeing measured, allowing for some small geometric distortions caused bythe lens 165. Computer 105 has been programmed to digitize the cameraimage and then by utilizing vision software determines a position withinwell A1 for the drop of liquid hanging from grease seal 361. Theposition of the drop of liquid is recorded for later use onto the harddrive of computer 105 and to a memory location within the computer.

In a preferred embodiment, the vision software used to determine theposition of the drop of liquid uses a software routine, algorithm calledmvt_blob_find from a collection of image processing tools calledMVTools. MVTools is available from Coreco Imaging, US Office in Bedford,Mass.

After recording the position of the drop of liquid hanging from greaseseal 361 in well A1, linear actuator 115 moves fixture plate slightly tothe left so that lens 165 is over well B1 (FIG. 13). In a fashionsimilar to that described for well A1, the position of the drop ofliquid hanging from grease seal 361 in well B1 is recorded on the harddrive of computer 105 and in computer memory. For example, as shown inFIG. 15, computer 105 will record that drop of liquid α1 is towards theupper left-hand quadrant of well A1. Likewise, the position of drop ofliquid β1 is recorded onto the database of computer 105 as being in thelower right-hand quadrant of well B1.

In this manner, positions of the drops of liquid are recorded for cellsA1-H12. In FIG. 18, linear actuator 115 has moved fixture plate 129 sothat well H1 is under lens 165.

In FIG. 19, linear actuator 115 has moved fixture plate 129 to the leftand linear actuator 150 has moved moving base 154 slightly rearward sothat lens 165 is over well A2 of micro-well plate 125A (FIG. 13).

In a manner similar to that described above, positions of the drops ofliquid are recorded for cells A2-H12 (FIG. 2, FIG. 13). In FIG. 20,linear actuator 115 has moved fixture plate 129 to the left and linearactuator 150 has moved moving base 154 rearward so that well H12 isunder lens 165.

After positions of the drops of liquid are recorded for cells A1-H12 formicro-well plate 125A, linear actuator 115 moves fixture plate 129 andlinear actuator 150 moves moving base 154 so that cell A1 of micro-wellplate 125B is underneath lens 165 (FIG. 21).

In a manner similar to that described above, positions of the drops ofliquid are recorded for cells A1-H12 for each micro-well plate125A-125F. In FIG. 22, linear actuator 115 has moved fixture plate 129and linear actuator 150 has moved moving base 154 so that well H12 ofmicro-well plate 125F is under lens 165.

Recording the Image of the Drop of Liquid within Each Well

An operation to inspect each hanging drop is performed at a highermagnification using camera 135 with its zoom lens 145 capable ofmagnifications of 2.5× to 10× corresponding approximately to digitizedpixels representing 2.68 microns square (at 2.5×) to 0.67 microns square(at 10×). This inspection is done for the purpose of determining whetherprotein crystals have grown. Zoom motor 192 controls the degree of zoomfor zoom lens 145. Using data representing the position of the drop ofliquid within each well obtained during the inspect-well sequence,computer 105 (FIG. 1) automatically transmits a signal to linearactuators 115 and 150 to position lens 145 directly over the drop ofliquid within each well. For example, in FIG. 23 lens 145 is positionedover the top of well A1 of micro-well plate 125A. Using the positioningdata earlier obtained, lens 145 is precisely positioned so that it isable to zoom in on drop of liquid α1 (FIG. 13). FIG. 16 shows amagnified view of drop of liquid α1. In FIG. 23, motor 130 of linearactuator 160 has moved moving plate 162 upward and/or downward asnecessary to properly focus lens 165 on drop of liquid α1. Zoom motor192 has manipulated lens 165 to obtain the desired degree of zoom.Camera 135 inspects well A1 and transmits a signal representing themagnified image of the hanging drop of liquid to computer 105. Theimages are stored on computer 105 temporarily in memory for immediateanalysis and on hard disk for later analysis.

In a similar fashion, linear actuators 115, 150 and 160 and zoom motor192 operator to properly position and magnify zoom lens 145 over eachhanging drop of liquid to obtain desired focus and magnification forimage data storage. For example, FIG. 17 shows a magnified view ofhanging drop of liquid β1.

In a manner similar to that described above during the inspect-wellsequence, magnified images of the drops of liquids (similar to thoseshown in FIGS. 16 and 17) are recorded for cells A1-H12 for micro-wellplate 125A (FIG. 2, FIG. 13). In FIG. 24, linear actuator 115 has movedfixture plate 129 and linear actuator 150 has moved moving base 154 sothat well H12 of micro-well plate 125A is under lens 165.

Then, the sequence is repeated for micro-well plates 125B-125F so thatmagnified images of the hanging drops of liquids are recorded for eachcell A1-H12 for micro-well plates 125B-125F. In FIG. 25, the sequencehas ended for micro-well plates 125A-125F. Linear actuator 115 has,movedfixture plate 129 and linear actuator 150 has moved moving base 154 sothat well H12 of micro-well plate 125F is under lens 145.

Manual Scoring the Drop of Liquid within Each Well

After micro-well plates 125A-125F have been run, monitor 620 will appearas shown in FIG. 26. In FIG. 26, six images representing micro-wellplates 125A-125F appear on the screen. Above each image is a message“Run Comp” indicating that image data for hanging drops of liquid hasbeen transferred into computer 105. Beneath each image are buttons710-715 marked “S”. By mouse clicking on any button 710-715, theoperator may manually score for successful crystal formation eachmagnified image of each hanging drop of liquid.

For example, in FIG. 26, the operator can mouse click on button 710 toscore micro-well plate 125A.

In FIG. 27, the operator has mouse clicked on the circle representingwell A1 of micro-well plate 125A. This has caused a magnified image tobe displayed of drop of liquid al in screen section 716. The operatorhas concluded that there are no crystals in drop of liquid α1 and hastherefore mouse clicked on button 717 for “NO CRYSTAL”. On the displayscreen, this has caused the circle representing well A1 of micro-wellplate 125A to turn red.

In FIG. 28, the operator has mouse clicked on the circle representingwell B1 of micro-well plate 125A. This has caused a magnified image tobe displayed of drop of liquid β1 in screen section 716. The operatorhas concluded that there are crystals in drop of liquid β1 and hastherefore mouse clicked on button 718 for “CRYSTAL”. On the displayscreen, this has caused the circle representing well B1 of micro-wellplate 125A to turn green.

In a similar fashion, the above scoring procedure is repeated until allwells A1-H12 for micro-well plates 125A-125F have been scored as eitherred (NO CRYSTAL) or green (CRYSTAL).

Data Utilization

Once micro-well plates 125A-125F have all been scored, the operator hasat his easy disposal a database that contains the identity of eachmicro-well plate that was inspected along with a score summarizingwhether crystal formation occurred for each well in the micro-wellplate. The automated and efficient manner in which the operator is ableto acquire his contrasts with the prior art method of laboriouslyinspecting each well with a microscope and the handwriting the resultsinto a notebook.

For example, to score six 96-well-micro-well plates utilizing thepresent invention should take approximately no more than 10 to 15minutes.

In contrast, the prior art method of inspecting six 96-well micro-wellplates with a microscope and the handwriting the results into a notebookwill take approximately 30 to 100 minutes depending on the conditionsdiscussed in the background section in addition to the time required totranscribe the results into a computer database. Plus, as previouslyexplained in the background section, manual inspection and scoring issubject to a relatively high risk of human error.

Second Preferred Embodiment

In a second preferred embodiment, the depth of view of camera 135 isapproximately 50 to 100 micrometers. The crystal in the drop of liquidmay be larger than the depth of view or there may be crystals growing atvarious levels within the hanging drop of liquid, as shown in FIG. 29.Therefore, in the second preferred embodiment, lens 145 is focused atmultiple different levels 721-724 and a set of images are recorded atthe different levels so that the entire crystal may be analyzed.

Specimen Auto-Focus

The third preferred embodiment of the present invention utilizes aspecimen auto-focus subroutine 300 (FIG. 31). Subroutine 300 ensuresthat the specimen within the micro-well is in-focus at the desired zoom(or magnification ratio) of image lens 145. Utilizing the auto-focusfeature, the present invention causes camera 135 to take a number ofimages defined by a Number_of_Z_Slices. Typically, there are between 5and 10 slices separated in the Z-axis from one another by a Z_Step_Size.The typical step size is 0.05 mm to 0.25 mm. The slices preferably startat a Z-Axis location defined by a Start_Z value, which is typically atthe bottom of the cover-slip on the micro-well plate. During specimenauto focus initialization 302, input data 304 is received. The area ofinterest window within the images is further defined in the input data304 by a X_Window_Center and a Y_Window_Center plus an X_Width and aY_Width. Initial settings 306 for the routine 300 are the starting valueof a counter N, a Best_Focus, a Best_Z, and a Focus_Error. Theinspection device sets Z_Position(N) equal to the Start_Z location andmoves camera 135 there in step 308. An image is acquired with, camera135 and digitized as previously described and stored as Image (N) instep 310. A second subroutine 312 extracts a focus value F (N) for Image(N) and is further described in the section for discussion for FIG. 32.A, test is made between F(N) and the Best_Focus in step 316, such thatif F(N) is greater than Best_Focus then Best_Focus is set to F(N) andBest_Z is set to N as shown in step 320 and the program flow goes ontostep 322, if the test condition is not met in step 316 then the programflow skips step 320 and goes on to step 322. In step 322, a test is madeto determine of all of the slices have been taken as N is tested againstthe Number_of_Z_Slices. If N is equal to Number_of_Z_Slices then programflow goes onto step 324. If more slice images are needed, then the flowgoes to step 318. In step 318, Z_Position(N+1) is set toZ_Position(N)+Z_Step_Size and the Z-Axis is moved to Z_Position(N+1) andthe program flow goes on to step 314 where N is incremented by 1 (one).The program flow goes back to step 310 and completes the loop of step310 to step 322 until all of the image slices have been taken and thenmoves onto step 324. In step 324, Best_Z is tested against its initialvalue, and if it equals its initial value (meaning no focus was found inthe focus value subroutine 312) then it is set to a default value of theNumber_of_Z_Slices divided by 2 and Focus_Error is set to 1 (one) instep 326 and the program flow goes onto step 328. If Best_Z in step 324has a value other than its initial value then program flow goes ontostep 328 from step 324. In step 328 a Best₁₃ Image image is set to theimage slice at best focus by setting Best_Image equal to Image(Best_Z).Also, a Best_Image_Z value is set equal to Z_Position(Best_Z) and theflow goes onto step 330 which is the RETURN part of the subroutine andprogram flow returns to the main software flow.

As illustrated in FIG. 32. an image(N) focus F(N) subroutine 312 isfurther detailed, starting at the step Start 401. A pointer_to_image(N)404 is provided in step 402. In step 408 the image(N) is convolved witha standard 3×3 Sobel_Horizontal filter 416 to produce an Image(N)_Hwherein horizontal edges within the image are emphasized. In step 410the image(N) is convolved with a standard 3×3 Sobel_Vertical filter 418to produce an Image(N)_V wherein vertical edges within the image areemphasized. In step 414, both the horizontal edge emphasized imageImage(N)_H and the vertical edge emphasized image Image(N)_V are summedpixel by pixel, during the summing process any resulting negative pixelvalues are set to zero and any resulting pixel values that are greaterthan 255 are set to 255, to produce an image Image(N)_Focus. In step420, a simple variance F(N) is calculated for the pixels within a windowof interest defined in 422 by X-Window_Center, Y_Window_Center, X_width,and Y_Height. The resulting value of the variance is returned to thecalling program as F(N) in step 424. The sobel processing and thevariance calculation is performed with a collection of image processingsoftware tools within MVTools. MVTools is available from Coreco Imaging,US Office in Bedford, Mass.

Fourth Preferred Embodiment

In the first preferred embodiment, it was disclosed how an operatorcould manually score each drop of liquid as either “CRYSTAL” or “NOCRYSTAL”. In the fourth preferred embodiment, the operator is given agreater variety of options in deciding on how to score each drop. Table1 shows listing of the operator's scoring options, including number,text description, and the corresponding color code. Once a micro-welldrop has been scored a 9 the operator can further classify the crystalsin a scoring shown in Table 2.

TABLE 1 SCORE DESCRIPTION DISPLAY COLOR 0 clear White 1 lightprecipitation Red 2 heavy precipitation Yellow 3 ugly precipitation Blue4 phase separation Orange 5 unknown Violet 6 Spherolites Black 7 Grainyprecipitation Gray 8 Microcrystals Brown 9 Crystal Green

TABLE 2 SCORE DESCRIPTION 9.0 crystal (no comments) 9.1 needles,intergrown 9.2 needles, single 9.3 plates, intergrown 9.4 plates, single9.5 chunks, <50 microns, intergrown 9.6 chunks, <50 microns, single 9.7chunks, >50 microns, intergrown 9.8 chunks, >50 microns, single 9.9gorgeous > 50 microns

Fifth Preferred Embodiment

In the fourth preferred embodiment, it was disclosed how an operator canmanually score each drop of liquid into one of 10 categories withcorresponding color coding, and how the operator can score category 9into further subcategories of 9.0 through 9.9. In the fifth preferredembodiment, the inspection device automatically scores and classifieseach drop specimen by executing computer software subroutines as shownin FIGS. 33, 34 a, 34 b, 34 c, 34 d, and 35 a and 35 b under control ofthe program flow shown in FIG. 36. The automatic classification canoccur at three levels of detail, the first level,Type_of_Classification=1, simply discriminates between a drop that isclear or not-clear (unknown), the second level,Type_of_Classification=2, scores and classifies the drop into classes 0through 9 as described in Table 1 above, and the third level,Type_of_Classification=3, performs second level scoring andclassification, plus adds an additional 10 subcategories to the CLASS 9,crystal classification, as detailed in Table 2 above.

Automatic Scoring and Classification

FIG. 36 illustrates the main program flow 840 starting at step 842. Thesoftware is initialized with parameters, inspection lists, andType_of_Classification detailed in step 486. The flow continues ontostep 844 where the system moves the micro-well of interest within theselected micro-well plate under the selected camera. In step 851, if thedrop needs to be located, the flow continues onto step 849 wherein animage is acquired by the camera and software operates on the image anddetermines the location of the drop. Then a test is made to determine ifthe last micro-well in the plate has been imaged, if not then the flowloops to step 844 and continues. If the last micro-well in plate step853 has been imaged then the flow continues to step 856 where the systemmoves to the droplet within micro-well in micro-well plate under thehigh-resolution camera. Also in step 851 if the drop had been previouslylocated, then the flow would continue from step 844 directly to step 856without the need to re-locate the drop. From step 856, the flowcontinues onto step 857 wherein a high-resolution image of the drop isobtained. Then the flow goes onto step 850. In step 850, a CALL is madeto subroutine that automatically scores and classifies the dropdepending on the Type_of_Classification required. The subroutine isdetailed in FIG. 33 and starts at step 440 in FIG. 33. After the drophas been classified the subroutine returns to step 852 wherein theresults are stored and reported. The program flow continues to step 858where a test is made to determine if the last drop in the selected platehas been processed, if so then the flow goes onto step 848 wherein atest is made to determine if the last plate has been processed. If not,the flow loops back to 856 and continues. If the last plate has beenprocessed, the flow goes onto step 854 and the program is done.

FIG. 33 shows Micro-well Specimen Auto Score and Classify Subroutine 438starting at step 440. Pointer_to_Best_Image 442 provides information tothe initialization step 444 that allows access to the image that wasfound to be the best focus. Plus, the Type_of_Classification is passedinto the routine. Alternatively, pointer 442 can point to an image thatwas taken at a z height value known to be the focus of the system. Afterinitialization 444 the subroutine 438 calculates in step 448 anaverage_gray_Background value normalized to allow for the variationpresent from various plate types, by using a first rectangular windowdefined by parameters shown-in step 446 (X_Background_Center,Y_Background_Center, X_Background_Width, and Y_Background Height) and bysumming all of the gray scale values of the pixels defined by the window446 and dividing by the number of pixels within that window. Theaverage_gray_background is normalized for well-type differences bymultiplying the calculated value by a Micro-well normer value also foundin step 446 and generally determined by measuring the various micro-wellplate types under inspection and normalizing to the 96-well standardmicro-well plate. This average_gray_background 448 is calculated in awindow area of the image that is outside the area of the drop butgenerally within the well or within the bounding well walls.

In step 450 an average_gray_Classify_Window value is calculated in asimilar manner as described above (except it is not normalized formicro-well type) using a second rectangular window defined by parametersshown in step 452 (i.e., X_Classify_Center, Y_Classify_Center,X_Classify_Width, and Y_Classify Height). Thisaverage_gray_classify_window value 450 is taken in a rectangular windowarea of the image that is inside the area of the drop and defined bybeing a fraction between 0.98 and 0.5 (with 0.8 preferred) of the widthand height of the external bounding rectangular box from the blobutilizing subroutine mvt_blob_find. The subroutine mvt_blob_find definesthe extent of the drop as previously discussed in the section“Determining the Position of the Drop of Liquid within Each Well”.

In step 454, a diagonal difference image is calculated by stepwisesubtracting pixel values from two pixel locations, defined by (x,y) and(x+Diff_Sep, y+Diff_Sep) within the Classify_window. The pixel valuesare separated in width and height by the value Diff_sep from step 452.This is repeated over all pixels within the Classify window defined bystep 452 using X_Classify_Center, Y_Classify_Center, X_Classify_Width,and Y_Classify Height. For each value calculated the absolute value istaken of the subtraction result and compared to a threshold valueFlag_Thresh defined in step 452. If the calculated value is greater thanFlag_Thresh 452 then the pixel is set at the first location in x,y equalto a value defined by Set1 in step 452, if the calculated value is equalto or less than Flag_Thresh, 452, then the pixel value is set to zero.This can be seen by the mathematical equations and flow described instep 454 in calculating a Diff_Image_1. Typical values for Diff_Sep arebetween 1 to 20 pixels with 7 preferred. Typical values for Set1 arebetween 1 and 511 with 128 preferred. Typical values for Flag_Thresh arebetween 5 and 50 with 25 preferred.

In Step 456, a calculation similar to that performed in step 454 isperformed on the Classify_Window except that the separation between thetwo pixels undergoing the calculation is defined by (x+Diff_Sep, y) and(x, y+Diff_Sep), as is shown in the mathematical calculation in 456 togenerate a Diff_Image_2. This calculation uses definitions shown in step452. Typical values for Set2 are between 1 and 511 with 200 preferred.

In Step 458 the Classify_Image, which is a combination of the imagesgenerated in step 454 and 456 is calculated as shown by the mathematicalequations shown in step 458 using definitions shown in step 452. If thex,y pixel value in either Diff_Image_1 or Diff_Image_2 (steps 454 and456 respectively) has a value equal to Set1 452 then the pixel value isset at (x,y) in Classify Image equal to Set2 452. Otherwise, the valueis equal to zero(0) as shown in the mathematical equations in step 458.The calculations are repeated for all pixels within the window definedin 452. The Classify_Image is basically an image of theclassify_image_window wherein edges present within the originalBest_Image are detected.

In step 460, the value of number_Pixels_Set2 is set equal to the totalnumber of pixels that are set equal to Set2 452 in step 458. Also, thevalue of Total_Pixels_in_Window is set to the total number of pixels inthe Classify_window in step 458.

In step 462, a Score_Gray is calculated by dividing theAverage_Gray_Classify_Window determined in step 450 by theAverage_Gray_Background found in step 448. A Score_Flag is, alsocalculated by dividing the Number_Pixels_Set2 by Total_Pixels_in_Windowfrom step 460. The Score_Gray and Score_Flag are normalized in thismatter.

In step 464, the values of Score_Gray, Score_Flag, andType_of_Classification are passed to a classify subroutine and aclassification is returned for the Classify_image, effectivelyclassifying the protein crystals within the window. Details of theClassify subroutine 464 are provided in FIGS. 34 a, 34 b, 34 c, and 34d, plus FIGS. 35 a and 35 b.

FIG. 34 a shows the Classify Subroutine 464. Start of Classifysubroutine is shown in step 468 followed by initialization 470 wherebythe initial classification CLASS value is set to 5, representing“unknown” and the flow goes onto step 480.

Step 480 calculates whether the drop is clear and the CLASS=0 by a testdetailed in step 480 using thresholds defined in step 482 (Clear_Flag_LTand Clear_Gray_GT) with the following equation: if score_flag is lessthan Clear_Flag_LT and Score_Gray is greater than Clear_Gray_GT then setCLASS=0. At step 481, a test is made to see of thetype_of_classification is equal to 1, the first classification typewherein the drop is classified as simply clear (0) or unknown (5) aspreviously discussed. If the type_of_classification is equal to 1 thenthe flow goes on to step 483 and then returns to FIG. 33 step 466 withthe results. If the type_of_classification is not equal to 1 then theflow goes onto step 476 for further classification.

Step 476, utilizing threshold value parameters shown in step 478(Lgt_Precip_Flag_LT, Lgt_Precip_Flag_GT, Lgt_Precip_Gray_GT), assignsthe value of 1 to CLASS indicating that Light Precipitation is presentin the Classify_Image. Step 476 utilizes the mathematical equation whichstates if Score_Flag is less than Lgt_Precip_Flag_LT and Score_Flag isgreater than Lgt_Precip_Flag_GT and Score_Gray is greater thanLgt_Precip_Gray_GT then set CLASS to value 1.

Step 484 calculates heavy precipation by using thresholds detailed instep 486 (Heavy_Precip_Flag_LT, Heavy_Precip_Flag_GT,Heavy_Precip_Gray_LT) with the following equation: if score_flag is lessthan Heavy_Precip_Flag_LT and score_flag is greater thanHeavy_Precip_Flag_GT and score_gray is less than Heavy_Precip_Gray_LTthen set CLASS=2.

Step 488 calculates ugly precipation by using thresholds detailed instep 490, Ugly_Precip_Flag_LT, Ugly_Precip_Flag_GT, Ugly_Precip_Gray_LT,with the following equation: if score_flag is less thanUgly_Precip_Flag_LT and score_flag is greater than Ugly_Precip_Flag_GTand score_gray is less than Ugly_Precip_Gray_LT then set CLASS=3.

Step 492 continues the classification process in FIG. 34 b.

In FIG. 34 b, the continuation of the classification process 700 isshown continuing in step 702. Step 704 calculates micro_crystals byusing thresholds detailed in step 703 (Micro_Cry_Flag_GT andMicro_Cry_Gray_LT) with the following equation: if score_flag is greaterthan Micro_Cry_Flag_GT and score_gray is less than Micro_Cry_Gray_LT,then set CLASS=8.

Step 704 calculates crystals by using thresholds detailed in step 705(Crystal_Flag_LT, Crystal_Flag_GT, and Crystal_Gray_GT) with thefollowing equation: if score_flag is less than Crystal_Flag_LT and scoreflag is greater than Crystal_Flag_GT and score_gray is greater thanCrystal_Gray_GT then set CLASS=9.

Step 706 calculates Grainy precipitation by using thresholds detailed instep 707 (Grainy_Flag_GT, and Grainy_Gray_GT) with the followingequation: if CLASS=8 and score_flag is greater than Grainy_Flag_GT andscore gray is greater than Grainy_Gray_GT, then set CLASS=7.

Step 708 continues the classification process onto FIG. 34 c as 725.

Step 726 continues from step 708 of FIG. 34 b.

In FIG. 34 c, Step 727 calculates and generates a first set ofadditional image features for further use in classification taking asinput 726 (X_Classify_Center, Y_Classify₁₃ Center, X_Classify_Width,Y_Classify Height, Pointer_to_classify_image, andmvt_blob_analysis_Params). In step 727, MVT_Tools_Blob_Analysis withpointer to window in Classify_Image is called. Step 727 gets thenum_found of blobs and, for each blob found, step 727 gets its area(m),height(m), width(m), perimeter(m), and location of each as location_X(m)ands location_Y(m). These values are recorded. These calculations areperformed on the image called Classify_Image, which is the image thatwas formed in FIG. 33 as step 458.

In step 728, if num-found is not greater than zero(0) the subroutinegoes to step 738. However, if any blobs are found then further analysisis started in step 729 by setting m=1, num_spherolite=0, andspherolite(m)=0. In step 730, the following blob_ratios are calculated:Circle_Like_HW(m)=height(m)/Width(m) andCircle_Like_AHW(m)=area(m)/(height(m)*width(m). For blobs that arecircular, Circle_Like_HW will be around a value of one (1). If blobs getelongated then the value will be other than one. Circle_Like_AHW forcircular blobs has a value around 0.785. For square-like blobs the valuewill be closer to one (1). The program flow goes onto step 732.

Step 732 determines whether to classify a drop as having a spherolite byutilizing the parameters found in 734 (Circle_Like_HW_lower,Circle_Like_HW_Upper, Circle_Like_AHW_Lower, and Circle_Like_AHW_Upper).The following equation is used:: if CLASS=8 or 9 and Circle_Like_HW(m)is less than Circle_Like_HW_Upper and Circle_Like_HW(m) is greater thanCircle_Like_HW_lower, and Circle_Like_AHW(m) is less thanCircle_Like_AHW_Upper and Circle_Like_AHW(m) is greater thenCircle_Like_AHW_Lower, then num_spherolite=num_sperolite+1. An incrementis done by calculating m=m+1 and setting spherolite(m)=1 to show one hasbeen found at m.

Step 736 tests whether all of the found blobs have been classified bytesting m against num_found and, if equal, the subroutine goes onto step738. If not the program loops back to step 730 and flows through asabove.

Step 738 goes onto FIG. 34 d in the classification subroutine as 740.

FIG. 34 d continues from FIG. 34 c at 742 and goes to 744 where a secondset of additional features is calculated. These features are generatedwith the original Best_Image generated in step 328 (FIG. 31) as theimage of the best focus. For these additional features, the results areused from the blob analysis performed in FIG. 34 c on theclassify_image. Such features as shown in parameters 746 as num_found,area(m), height(m), width(m), perimeter(m), location_X(m),location_Y(m), num_spherolite, and spherolite(m). Plus it uses theaverage_gray background from FIG. 33 step 448, and thepointer_to_Best_Iimage in FIG. 31 step 328. In step 744 “m” andnum_phase_sep are set equal to zero.

In step 748, if the number of spherolites previously found(num_spherolite) is not greater than zero (0), then the flow goes tostep 756 and the remaining steps shown in FIG. 34 d are bypassed: But ifthe num_spherolite is greater than zero (0), then the flow goes ontostep 749 wherein m is incremented by 1. Then the flow goes on to step751 to test whether to go to step 756 or to step 753 depending oncomparing “m” to the num_found. In step 753, the value of spherolite(m)found in FIG. 34 c step 732 is tested to see if any were classified asspherolite. If not, then the flow loops back to step 749. Ifspherolite(m) is equal to one (1) then step 750 is executed.

In step 750, the information location_X(m), location_Y(m), one half ofheight(m) and one half of width(m) is used to calculate the average grayvalue within this reduced image inside each spherolite. The Best_Imagepixel data obtained from the previous blob analysis is utilized andphase_sep_Gray(m) is set equal to this value. Phase_sep_Gray(m) is thennormalized by dividing it by the Average_Gray_Background. The programflow goes onto step 752, wherein phase_sep_Gray(m) is tested to see ifit is greater than a parameter phase_sep_Gray_GT from parameter inputstep 754. If true, then CLASS is set equal to four(4) and the programloops up to step 749.

In step 751 the automatic program has classified the microdrop into oneof the 10 primary classes, 0 through 9, detailed previously in Table 1.Then, a test is made to see if the type_of_classification is equal to 2.If so, then the flow goes onto step 755 wherein the flow is returned toFIG. 33. In step 466, a general classification is complete. If furtherclassification into subcategories is required, then the flow goes ontostep 758.

In step 758, the program flow goes onto FIG. 35 a step 760.

In FIG. 35 a, a crystal classification subroutine 762 further classifiesany CLASS 9 crystal image into to additional subclasses, 9.2, 9.4, 9.6,9.8, or 9.9, as previously discussed in Table 2. The subroutine beginsat step 760 and goes onto step 764. In step 764, the blob counter m isset to zero. A test is conducted to see if the CLASS is equal to 9 and,if not, the flow goes to step 778 and returns to FIG. 33 step 466. Step764 uses input values as shown in step 766 (Average_Gray_Background,Pointer_To_Best_Image, num_found, area(m), height(m), width(m),perimeter(m), location X(m), location Y(m), num_spherolite, andspherolite(m)), as previously described. If the CLASS is equal to 9another test is made to see if one or more than one blobs werepreviously found. If only one blob was previously found then the flowgoes onto step 768. If more then one is found, the flow goes onto step769 in FIG. 35 b. In step 768, a height to width ratio is calculated andcompared to thresholds representing needle-like characteristics in step770. If the test conditions are met, then the CLASS is set to 9.2. Ifnot, the flow goes onto step 771. In step 771, the height to width ratiois compared to thresholds representing plate-like characteristics instep 772. If the conditions of the test are met, the CLASS is changed to9.4 and the flow goes onto step 773. If not, no further classificationis performed and the flow goes onto step 778 and the subroutine returnsto FIG. 33 step 466. In step 773, a normalized average gray value,Chunk_Gray(m), within the blob is calculated and the flow goes onto step774. In step 774, if Chunk_gray(m) is less than a thresholdChunck_Gray_LT (from step 775), then CLASS is set to 9.6 and the flowgoes onto step 776. In step 776, if Chunk_gray(m) is greater than athreshold Chunk_50_GT, (from step 777), then CLASS is set to 9.8 and theflow goes onto step 780. In step 780 if area(m) is greater thanthreshold gorgeous_GT (from step 779), then CLASS=9.9 and the flow goesonto step 778. Then the subroutine returns to FIG. 33 step 466.

FIG. 35 b shows flowchart 781 illustrating the further classification ofa CLASS 9 image having multiple blobs within the image into additionalsubclasses, 9.1, 9.3, 9.5, or 9.7, as previously discussed in Table 2.The subroutine begins at step 786 from step 769 in FIG. 35 a and goesonto step 782. In step 782, a loop begins at m equals 0 for each blob musing input values from step 784 (Average_Gray_Background,Pointer_To_Best_Image, num_found, area(m), height(m), width(m),perimeter(m), location_X(m), location_Y(m), num_spherolite, andspherolite(m)) as previously described. The flow goes onto step 788where m is incremented and a height to width ratio is calculated andcompared to thresholds representing needle-like characteristics in step790. If the test conditions are met, then the CLASS is set to 9.1. Ifnot, the flow goes onto step 794. In step 794, the height to width ratiois compared to thresholds representing plate-like characteristics instep 796. If the conditions of the test are met, the CLASS is changed to9.3. If not, no further classification is performed and the flow goesonto step 812 to determine if all of the blobs have been tested. In step798, a normalized average gray value (Chunk_Gray(m) as described before)within the blob is calculated and the flow goes onto step 802. In step802, if Chunk_gray(m) is less than a threshold Chunk_Gray_LT (from step804) then CLASS is set to 9.5 and the flow goes onto step 806. In step806, if Chunk_gray(m) is greater than a threshold Chunck_50_GT (fromstep 808) then CLASS is set to 9.7 and the flow goes onto step 812. Instep 812, if all of the blobs have been tested, the flow goes onto step810 and the subroutine returns to FIG. 33 step 466. If not all tested,then the flow loops back to step 788 for the next blob and the processloops until complete.

Typical parameter and threshold values for use in classification inFIGS. 33, 34 a, 34 b, 34 c, 34 d, and 35 a and 35 b are given in Table 3along with the preferred value. These values serve only as a guide, andother values may be used when circumstances justify. For example,different lighting conditions, variations in the transparency ofmicro-well plates, variations in the formulations of the protein growingmedia and drop, values may be used as well. One skilled in the art mayadjust the-parameter and threshold values to tune in the classificationresults specific to their setup.

TABLE 3 Typical and preferred values for threshold and Classificationparameters Step Name lower value upper value Preferred value 452Diff_Sep 1 20 7 452 Set1 1 511 128 452 Set2 1 511 200 452 Flag_Thresh 550 25 478 Lgt_Precip_Flag_LT 0.0001 0.010 0.002 478 Lgt_Precip_Flag_GT0.0 0.001 0.00001 478 Lgt_Precip_Gray_GT 0.8 1.2 1.020 482 Clear_Flag_LT0.000001 0.01 0.0007 482 Clear_Gray_GT 0.8 1.2 1.020 486Heavy_Precip_Flag_LT 0.0001 0.050 0.020 486 Heavy_Precip_Flag_GT 0.00.005 0.001 486 Heavy_Precip_Gray_LT 0.7 1.2 0.95 490Ugly_Precip_Flag_LT 0.0001 0.050 0.020 490 Ugly_Precip_Flag_GT 0.0 0.0050.001 490 Ugly_Precip_Gray_LT 0.5 1.2 0.7 703 Micro_Cry_Flag_GT 0.00010.05 0.020 703 Micro_Cry_Gray_LT 0.9 1.20 1.010 705 Crystal_flag_LT 0.10.5 0.30 705 Crystal_Flag_GT 0.001 0.05 0.01 705 Crystal_Gray_GT 0.8 1.21.0099 707 Grainy_Flag_GT .001 .5 0.177 707 Grainy_Gray_GT 0.8 1.2 0.980734 Circle_Like_HW_lower 0.5 1.00 0.9 734 Circle_Like_HW_Upper 1.00 2.01.1 734 Circle_Like_AHW_lower 0.1 3.9 0.78 734 Circle_Like_AHW_Upper.785 2.0 0.83 754 phase_sep_Gray_GT 0.9 1.2 1.05 770, 790 needle_HW_GT 220 5 770, 790 needle_HW_LT 0.5 0.05 0.2 772, 796 plate_HW_GT 1.1 2.00.95 772, 796 plate_HW_LT 0.9 1.1 1.05 775, 804 Chunk_Gray_LT 0.8 1.21.01 777, 808 Chunk_50_GT 70 200 100 779 Gorgeous_GT 7000 2000 1000

Prototype

Applicants have designed, built and tested a working prototype of thepresent invention.

FIG. 30 shows the major components of the Applicant's prototype.Proteomic crystal verification and inspection system 100 has three axisof linear motion. Linear actuator 115 is preferably linear actuatormodel # 802-0763D available from Dynamic Solutions of Irvine Calif.,with 600 mm of travel driven by an enclosed 0.5 mm pitch ballscrew.Linear actuator 115 is driven by an intelligent self-contained servomotor 110 model # SM2320 SQ available from Animatics Corp. of SantaClara, Calif., with 38 oz-in of available torque. Servo motor 110communicates with a Windows Based computer 105 through a serialconnection routed through a central control unit 190.

Linear actuator 115 has stationary part 116 fixed to a granite table top170. Motor 110 moves moving part 117 along the x-axis. Granite top 170is supported by a frame 180. Frame 180 has casters and adjustable legs.Plate 127 is attached to moving part 117. At each end of plate 127, aspacing block 128 spaces fixture plate 129 from plate 127. At each ofits ends, fixture plate 129 is supported by spacing block 128. Fixtureplate 129 provides for the mounting, removal, and positioning ofmultiple micro-well plates 125A-125F. Preferably micro-well plates125A-125F are agar plates, micro-titer well plates of 96, 384, 1536wells, or other sample plates.

Support 191 is positioned adjacent to fixture plate 129 and contains alight source. In the preferred embodiment, the light source is model #A08925 fiber-optic back light available from Aegis Electronics ofCarlsbad, Calif.

In the preferred embodiment, linear actuator 150 is model # 802-1384Aavailable from Dynamic Solutions of Irvine Calif., with 350 mm of traveldriven by an enclosed 0.5 mm pitch ballscrew. Linear actuator 150 hasstationary part 153 horizontally bridged above linear actuator 115 andsupported by pillars 152. Moving base 154 provides a mounting base forcomponents of linear actuator 160. In a preferred embodiment, linearactuator 150 is driven by driven by an intelligent self-contained servomotor 120 identical to motor 110. Servo motor 120 communicates withWindows Based computer 105 through a serial connection routed throughcentral control unit 190.

In a preferred embodiment, linear actuator 160 is model # 802-0756Davailable from Dynamic Solutions of Irvine Calif., with 100 mm of traveldriven by an enclosed 0.5 mm pitch ballscrew. Linear actuator 160 haswith a stationary part 161 mounted perpendicular to and fixed to movingbase 154 of linear actuator 150. Linear actuator 160 is driven byintelligent self-contained servo motor 130. Servo motor 130 ispreferably identical to motor 110. Servo motor 130 communicates with theWindows Based computer 105 through a serial connection routed throughthe central control unit 190.

Moving plate 162 provides a mounting base for camera 155. In a preferredembodiment camera 155 is a high-resolution monochrome megapixel CCDcamera model # CVM1 from JAI America INC. of Laguna Hills, Calif. Camera155 has lens 165. Preferably, lens 165 has a 0.75× to 3× zoom capabilityand is model # NT52-571 available from Edmund Industrial Optical ofBarrington, N.J. Moving plate 162 also provides a mounting base forcamera 135. Preferably, camera 135 is a high-resolution monochromemegapixel CCD camera model # CVM1 from JAI America INC. of Laguna Hills,Calif. Camera 135 has lens 145. Preferably, lens 145 has a 2.5× to 10×zoom capability and is model # NT54-396 available from Edmund IndustrialOptical of Barrington, N.J.

Moving plate 162 also provides a mounting base for a zoom lens motor192, an intelligent self-contained servo motor model # SM2315D availablefrom Animatics Corp. of Santa Clara, Calif., with 20 oz-in of availabletorque. The servo motor communicates with the Windows Based computer 105through a serial connection routed through the motion control box 190.The zoom lens motor operates the zoom lens 145 through a conventionalbelt drive.

A bar-code reader 175 is mounted adjacent linear actuator 115 and isattached to support 172 fixed on granite top 170. Bar-code reader 175 ispositioned to view a bar-code identity label attached to a micro-wellplate when the micro-well plate is positioned under linear actuator 150.Preferably,-bar-code reader 175 is model # BL601 available from KeyenceCorporation of America of Newark, N.J.

A plate sensor transmitter/receiver 186 is also fixed to support 172 andis aligned to sense whenever a micro-well plate 125 breaks a beam oflight emitted by transmitter/receiver 186 and reflected-by a reflector188. Reflector 188 is mounted on support 191 on the opposite side of thelinear actuator 115. Plate sensor transmitter/receiver 186 and reflector188 are preferably model # E3T-SR13 available from Western SwitchControls of Santa Ana, Calif.

Block Diagram Showing Connectivity of the Prototype

FIG. 6 shows a block diagram illustrating the connectivity ofApplicant's prototype. Linear actuator motors 110, 120 and 130, and zoommotor 192 receive DC power from 48 Volt DC power supply 680 through anelectrical connection. Linear actuator motors 110, 120 and 130, and zoommotor 192 motors communicate with Windows-based computer 105 throughcommon serial line 633. Bar-code reader 175 communicates with computer105 through a communications line and plate sensor 186 communicates withthe computer 105 through a communications line. Monitor 620 displaysinformation to the operator transmitted from computer 105. Cameras 135and 155 communicate with frame grabber 630 through communication lines.Frame grabber 630 is installed within computer 105 and is preferablyPCVision from Coreco Imaging US Office in Bedford, Mass. Frame grabber630 digitizes the image from the camera to form the digitized image dataarray within computer 105. A 4-port Ethernet hub 640 provides forconnectivity between computer 105, central control unit 190, and anexternal Ethernet 670. By providing for connectivity to the Ethernet,computer network communications are possible. The central control unit190 controls light source 194 through an analog control line. Centralcontrol unit 190 receives 24 volt DC power from 24 volt DC power supply660. Emergency stop button and switch (e-stop) 650 is connected tocentral control unit 190.

Experimental Results

Fifty-three test images were obtained from the system and were bothautomatically classified by the system and were manually classified byfour scientists. Table 4 shows the correlation percentage between thevarious scientists and the automatic classification provided by thesystem.

TABLE 4 Sam Mary Susan Fred AUTO Sam 100% Mary 98% 100% Susan 93% 97%100% Fred 89% 93% 96% 100% AUTO 95% 93% 86% 81% 100%

Utilization of Color

In another preferred embodiment, the present invention is configured torecord color images. It is desirable to be able to analyze color imagesin that certain precipitation products in the protein crystallizationprocess have distinctive colors and a crystallographer or automatedimage analysis algorithm may use the color information to helpdiscriminate crystallization results.

True Color Picture

FIG. 37 shows a side view of micro-well plate 125A positioned on fixtureplate 129. Support 191 with embedded light source 194 is positioned tothe side of fixture plate 129. Light from light guide 195 is directedupward through cutout 131. Light guide 195 is positioned between fixtureplate 129 and plate 127 such that both plates can move around the lightguide 195 without interference.

Linear polarized filter 352 is rotationally mounted above light guide195 such that light from light guide 195 can be polarized linearly at aprogrammable angle before it transits through micro-well plate 125A.Polarizer drive belt 356 (top view shown in FIG. 38) rotates polarizedfilter 352 about a vertical axis. Polarization drive belt 356 is drivenby motor 358. Motor 358 is controlled by CCU 190 (FIG. 40). Secondfilter 354 (top view shown in FIG. 39) is positioned above micro-wellplate 125A such that light transiting through micro-well plate 125A goesthrough second filter 354 before it goes into the camera zoom lens 145.Second filters 354 are mounted on filter wheel apparatus 355. Filterwheel apparatus 355 rotates the operator selected second filter 354 intoposition under the zoom lens 145. The selected second filter 354 ispreferably either a red, green or blue dichroic filter. Preferably,individual images taken through the red, green and blue second filtersare combined to form a true color image.

False Color Image

In addition to the true color images that may be formed using red,green, and blue filters 354, a false color (also called a pseudo color)image may be formed by taking three individual images using linearpolarized filter 352 at three different polarization angles with respectto a second filter. In this preferred embodiment, a linear polarizedfilter is substituted for the dichroic second filters 354 discussedabove. For example, the polarized axis may be at 90 degrees to eachother, and at plus and minus 45 degrees to each other. The three imagesare then called red, green, and blue and a false color image isproduced. If the crystal exhibits any polarization rotation effects,then a very colorful image results. This pseudo color image is useful indetecting very small and fine crystals from the image backgroundmaterial. Other polarizing angles may be selected as well.

In the preferred embodiment, light guide 195 is model # A08925fiber-optic backlight available from Aegis Electronics of Carlsbad.Calif. Light source 194 is a Dolan-Jenner Model-PL-900 available fromEdmund Industrial Optics, Barrington, N.J. First polarized filter 352,second filter 354 (including the linear polarized filters, and thedichroic red, green, blue filters) are available from Edmund IndustrialOptics, Barrington, N.J. Filter wheel 355 is model FW1 available fromIntegrated Scientific Imaging Systems, Inc of Santa Barbara, Calif.

Combined with an Automated Tray Storage and Retrieval System

In another preferred embodiment, proteomic crystal verification andinspection system 100 (FIG. 30) is operated in conjunction withautomated storage and retrieval system 900, as shown in FIGS. 41A and41B. FIG. 41A shows a front view and FIG. 41B shows a back view of theprototype automated storage and retrieval system 900 that Applicant'shave built and tested. Also show are two proteomic crystal verificationand inspection systems 100. In the preferred embodiment, an operatorloads micro-well plate holding tray 901 a (FIG. 42) into access drawer902 (FIG. 41A). Preferably, micro-well plate holding tray 901 a (FIG.42) holds 6 micro-well plates that contain solution in which proteincrystals are growing. Storage gantry 903 (FIG. 41B) removes tray 901 afrom access drawer 902. As shown in FIGS. 41A and 41B, automated storageand retrieval system 900 has a large number of storage slots. Afterremoving tray 901 a, storage gantry 903 transfers tray 901 a to one ofthe storage slots. Preferably, this process is repeated until aplurality of micro-well plate holding trays are stored in a plurality ofstorage slots. Then, at defined time intervals, storage gantry 903sequentially transfers micro-well plate holding trays from the storageslots to work cell area 904. An enlarged perspective view showing workcell area 904 is shown in FIG. 43. Once a micro-well plate holding trayhas been placed in work cell area 904, work cell gantry 905 willsequentially remove micro-well plates from the tray and place them ontoeither one of the two proteomic crystal verification and inspectionsystems 100. Each well in the micro-well plates will then be inspectedin a fashion similar to that described in the above preferredembodiments. After the micro-well plates have been inspected, work cellgantry 905 will place the micro-well plates back into their micro-wellplate holding trays. Then, storage gantry 903 will return the micro-wellplate holding trays back to their storage slots.

Sequence of Operation Storing the Mircowell-Plates

FIG. 45 shows a front view similar to the perspective view shown in FIG.41A. Access drawer 902 is closed.

FIG. 46 shows a top view of access drawer 902. In FIG. 46, an operatorhas opened access drawer 902.

In FIG. 47, the operator has placed trays 901 a and 901 b onto accessdrawer 902. Trays 901 a and 901 b each contain six 96-well micro-wellplates. Inside each well of each micro-well plate is a solution in whichthe operator is attempting to grow protein crystals.

In FIG. 48, the operator has closed access drawer 902. Sensors 922 a and922 b are situated adjacent to drawer 902 are directed to sense thepresence of a tray on access drawer 902. In the preferred embodiment,sensors 922 a and 922 b are MC-S series safety interlock switch, controlunit and sensors manufactured by Scientific Technologies Inc. withoffices in Freemont, Calif. Upon sensing the presence of trays 901 a and901 b, sensors 922 a and 922 b send signals to CPU 921 (FIG. 44). CPU921 is a windows based computer that has been programmed to operateautomated storage and retrieval system 900.

FIG. 49 shows a back view of automated storage and retrieval system 900similar to the perspective view shown in FIG. 41B. In FIG. 49, trays 901a and 901 b are resting on access drawer.

In FIG. 50, CPU 921 (after having received the signals from sensors 922a and 922 b) has sent a signal to linear actuator motor 923 a (FIG. 44).Linear actuator motor 923 a is connected to linear actuator 923 bdirectly and to linear actuator 923 c through actuator drive shaft 923d. In the preferred embodiment, linear actuators 923 b and 923 c arebelt driven linear actuators, Series HLE manufactured by ParkerAutomation-Daedal Division with offices in Irwin, Pa. In FIG. 50, linearactuators 923 b and 923 c have moved storage gantry 903 to the leftabove tray 901 a.

In FIG. 51, CPU 921 has sent a signal to linear actuator motor 924 a.Linear actuator motor 924 a is connected directly to belt driven linearactuator 924 b. In the preferred embodiment, linear actuator 924 b isSeries HLE manufactured by Parker Automation-Daedal Division withoffices in Irwin, Pa. Linear actuator 924 b controls the vertical motionof platform 925. Platform 925 is also supported by support bearing 924c. In FIG. 51, linear actuator 924 b has lowered platform 925 so that isapproximately adjacent to tray 901 a. Note that as platform 925 haslowered, counterweight cylinder 926 has risen. Counterweight 926 rideson bearing 927 and is connected to platform 925 by cable 928. Cable 928runs through pulleys 929 a and 929 b. In the preferred embodimentplatform 925 weights approximately 70 lbs. (fully loaded) andcounterweight 926 weights approximately 50 lbs. By connectingcounterweight 926 to platform 925, linear actuator 924 b has to do lesswork to move platform 925. Therefore, the vertical movement of platform925 is smoother.

FIG. 52 shows a top view of the position of platform 925 in FIG. 51.Linear actuator motor 930 a is connected directly to linear actuator 930b. Linear actuator motor 930 a and linear actuator 930 b are bothsupported by platform 925. In the preferred embodiment, linear actuator930 b is a gear driven linear actuator, ER Ball-screw seriesmanufactured by Parker Automation-Daedal Division with offices in Irwin,Pa. Linear actuator 930 b controls the horizontal movement of roboticgripper motor 931 a and robotic gripper 93 b. Robotic gripper motor 931a controls robotic gripper 931 b.

In FIG. 53, CPU 921 (FIG. 44) has sent a signal to linear actuator motor930 a, causing linear actuator 930 b to move robotic gripper 931 bcloser to tray 901 a. Robotic gripper has gripped tray 901 a.

In FIG. 54, robotic gripper 931 b has pulled tray 901 a along tracks 933onto platform 925. Bar code reader 934 is attached to the side ofplatform 925 and is orientated so that it is able to read the bar codeof each micro-well plate as it passes by. The bar code information istransmitted to CPU 921 where it is stored in the computer's database(FIG. 44).

In FIG. 55, storage gantry 903 has moved tray 901 a to its storage slotin storage rack 932. While at the location shown in FIG. 55, linearactuator motor 930 a moves robotic gripper 931 b (FIG. 52) towardsstorage rack 932 so that tray 901 a is stored inside its storage slot.

In FIG. 56, after having deposited tray 901 a in its storage slot,storage gantry 903 has moved adjacent to tray 901 b.

In FIG. 57, storage gantry 903 has moved trays 901 a-901 aab to theirassigned storage slots in storage rack 932 in a fashion similar to thatdescribe above in reference to FIGS. 44-56. Although FIG. 57 shows justfifty-four trays actually in storage rack 932, in the preferredembodiment, storage rack 932 can hold a total of 1692 trays. Also, inthe preferred embodiment, each tray can hold six micro-well plates.Therefore, in the preferred embodiment, storage rack 932 can hold atotal of 10,152 micro-well plates. Moreover, as previously noted inreference to FIG. 54, CPU 921 records the bar code of each micro-wellplate as its being loaded onto platform 925. Also, in the preferredembodiment, CPU 921 records the location of the storage slot in whicheach micro-well plate has been located. Therefore, for example, anoperator can immediately ascertain the location of a particularmicro-well plate by referring to the database of CPU 921.

Moving the Micro-well Plates to the Work Cell Area

In the preferred embodiment, by inserting commands into CPU 921 viakeyboard 934 (FIG. 44) an operator can inspect any given micro-wellplate at any moment. Or, CPU 921 can be programmed to automaticallycause automated storage and retrieval system 900 (FIG. 58) to transfer adesignated micro-well plate to proteomic crystal verification andinspection systems 100 at a specific designated time or-at designatedtime intervals. For example, in a preferred embodiment, CPU 921 has beenprogrammed to cause micro-well plates stored on trays 901 a-901 d to beinspected every two weeks for protein crystal growth without anyoperator intervention.

In FIG. 58, storage gantry 903 has moved to the top of tray stack 935containing trays 901 a-901 aab. While at this location, micro-well plate901 a will be loaded onto platform 925 in a fashion similar to thatdescribed above.

In FIG. 59, storage gantry 903 has moved platform 925 containingmicro-well plate 901 a in front of work cell area 904.

FIG. 60 shows a top view of platform 925 in front of work cell area 904.Work cell area 904 contains platform 936. At the end of platform 936 arerobotic grippers 937 b having robotic gripper motors 937 a. Roboticgripper motors 937 a are controlled by CPU 921 (FIG. 44). Adjacent toplatform 936 are fixture plates 129 of proteomic crystal verificationand inspection systems 100. A preferred proteomic crystal verificationand inspection system 100 is discussed above and shown in greater detailin FIG. 30. Work cell gantry 905 surrounds work cell area 904.

In FIG. 61, linear actuator 930 b has moved tray 901 a onto platform936. Robotic gripper 937 b has gripped the forward end of tray 901 a

In FIG. 62, robotic gripper 924 b has released the back end of tray 901a so that tray 901 a has been left in position on platform 936.

In FIG. 63, trays 901 b, 901 c and 901 d have been stored on platform936 in a fashion similar to that described in reference to tray 901 a.

In FIG. 64, linear actuator 938 b of work cell gantry 905 hasrepositioned linear actuator 939 b. Linear actuator motor 938 a isconnected to linear actuator 938 b directly and to linear actuator 938 dthrough actuator drive shaft 938 c. In the preferred embodiment, linearactuators 938 b and 938 d are belt driven linear actuators, model numberSeries HLE manufactured by Parker Automation-Daedal Division withoffices in Irwin, Pa.

In FIG. 65, linear actuator 939 b has moved gripping device 941 over amicro-well plate stored on tray 901 a. Linear actuator 939 b iscontrolled by linear actuator motor 939 a. In the preferred embodiment,linear actuator 939 b is a gear driven linear actuator, ER Ball-screwSeries manufactured by Parker Automation-Daedal Division with offices inIrwin, Pa.

FIG. 66 shows a side view of the situation in work cell area 904 shownin FIG. 65. Gripping device 941 is positioned over a micro-well platestored on tray 901 a. Gripping device 941 has linear actuator 942 bcontrolled by linear actuator motor 942 a. In the preferred embodiment,linear actuator 942 b is a gear driven linear actuator, ET Ball-screwSeries manufactured by Parker Automation-Daedal Division with offices inIrwin, Pa. Linear actuator 942 b controls the vertical movement ofrobotic gripper 943 b. The gripping movement of robotic gripper 943 b iscontrolled by robotic gripper motor 943 a.

In FIG. 67, linear actuator 942 b has lowered robotic gripper 943 btowards tray 901 a. Gripper 943 b has gripped the micro-well plate ontray 901 a.

In FIG. 68, gripper 943 b has removed the micro-well plate from tray 901a.

In FIG. 69, linear actuator 939 b has moved robotic gripping device 941so that the micro-well plate is positioned over fixture plate 129.

In FIG. 70, gripper 943 b has lowered the micro-well plate onto fixtureplate 129.

In FIG. 71, gripper linear actuator 942 b has raised gripper 943 b,leaving behind the micro-well plate on fixture plate 129.

FIG. 72 shows a top view of the situation depicted in FIG. 71. Roboticgripping device 941 is positioned over fixture plate 129.

In FIG. 73, all the micro-well plates stored on tray 901 a have beentransferred to fixture plate 129.

In FIG. 74, fixture plate 129 is moving the micro-well plates underneathlens 944 of proteomic crystal verification and inspection system 100.The operation of fixture plate 129 and proteomic crystal verificationand inspection systems 100 is described in detail above in reference toFIGS. 1-40. Robotic gripping device 941 has moved so that it isdepositing a micro-well plate stored on tray 901 d onto the secondfixture plate 129.

In FIG. 75, first fixture plate 129 has continued sequentially movingmicro-well plates under lens 944. Robotic gripping device 941 is in theprocess of depositing micro-well plates on second fixture plate 129.

In FIG. 76, second fixture plate 129 is sequentially moving micro-wellplates under lens 944. Robotic gripping device 941 is in the process ofremoving micro-well plates from first fixture plate 129 and placing themback onto tray 901 a.

In FIG. 77, all micro-well plates on trays 901 a through 901 d have beeninspected by proteomic crystal verification and inspection system 100 inthe manner described above: All micro-well plates are now back on trays901 a through 901 d. Storage gantry 903 (FIG. 41B and FIG. 49) will nowmove trays 901 a through 901 d back to their original positions instorage rack 932 (FIG. 57) until they are scheduled to be inspectedagain in accordance with programming input into CPU 921 (FIG. 44).

Operator Retrieval of Stored Tray

In the preferred embodiment, an operator has the option of personallyinspecting or handling a micro-well plate that has been stored instorage rack storage rack 932 (FIG. 55). The operator first inputs themicro-well plate's identifying bar code into CPU 921 via keyboard 934(FIG. 44). CPU sends a signal to storage gantry 903 (FIG. 41B, FIG. 59)and the appropriate tray containing the requested micro-well plate isremoved from storage rack 932. Storage gantry 903 then delivers the traycontaining the requested micro-well plate to access drawer 902 (FIG.41A, FIG. 46). The tray is loaded onto the access drawer 902. Once thetray is on access drawer 902, the operator can personally retrieve thetray by opening the drawer, as shown in FIG. 46. After he is finishedwith the tray, the operator returns the tray by placing it back onaccess drawer 902. Tray sensors 922 a and 922 b (FIG. 48) will sense thepresence of the tray and CPU will send a signal to storage gantry 903 togrip the tray. Bar code reader 934 (FIG. 54, FIG. 44) will read the barcode of the micro-well plates on the tray and storage gantry 903 willreturn the tray to its appropriate storage slot in storage rack 932.

Alternate Proteomic Crystal Verification and Inspection System

FIG. 83 shows another preferred embodiment of the present inventionhaving alternate work cell area 950. In the preferred embodiment, workcell area 950 includes two preferred proteomic crystal verification andinspection systems 951. A simplified perspective drawing of a preferredproteomic crystal verification and inspection system 951 is shown inFIG. 84 and a detailed perspective drawing of a preferred proteomiccrystal verification and inspection system 951 is shown in FIG. 85 a.

Operation of Alternate Proteomic Crystal Verification and InspectionSystem

The operation of proteomic crystal verification and inspection system951 can be understood by referring to FIGS. 84, 85 a and 85 b.Preferably, proteomic crystal verification and inspection system 951 hascamera positioning systems 955 a and 955 b. Computer 960 controls camerapositioning systems 955 a and 955 b so that cameras 959 a and 959 b aremoved along the x, y and z axis. Consequently, cameras 959 a and 959 bcan be focused over any well on micro-well plates positioned on tray 901a. For example, linear actuator 956 a moves camera 959 a along the xaxis, linear actuator 957 a moves camera 959 a along the y axis, andlinear actuator 958 a moves camera 959 a along the z axis. Likewise,linear actuator 956 b moves camera 959 b along the x axis, linearactuator 957 b moves camera 959 b along the y axis, and linear actuator958 b moves camera 959 b along the z axis. In the preferred embodiment,storage gantry 903 (FIG. 83) removes trays holding micro-well platesfrom storage rack 932 in a fashion similar to that described above.Storage gantry 903 then takes the removed tray to work cell area 950 forverification and inspection. While at work cell area 950 storage gantry903 will load the removed tray onto indexer 952. FIG. 85 a shows roboticgripper 931 b of storage gantry 903 (FIG. 83) placing tray 901 a ontoindexer 952. The micro-well plates on the removed tray will then beinspected by proteomic crystal verification and inspection system 951.

For example, in FIG. 84 robotic gripper 931 b (FIG. 85) has released itsgrip on tray 901 a so that tray 901 a is at the left side of indexer952.

A detailed perspective-view of indexer 952 is shown in FIG. 85 b.Indexer 952 includes drive system 980. Drive wheel 975 is controlled bycomputer 960. As drive wheel 975 turns, belt 976 turns wheel 977. Wheel977 is rigidly connected to the same axis as wheel 978. As wheel 977turns, so does wheel 978. The turning of wheel 978 causes belt 962 tomove. Bars 963 a and 963 b are rigidly connected to belts 962 and aremoved when belt 962 moves.

In FIG. 86, drive wheel 975 (not shown) has turned causing belt 962 tomove to the right. Bars 963 a and 963 b are rigidly connected to belt962 and have also moved to the right. Bar 963 a has pushed tray 901 a tothe right so that micro-well plate 901 a 1 is between camera 959 b andLED light source 965 b and micro-well plate 901 a 4 is between camera959 a and LED light source 965 a. In the preferred embodiment LED lightsources 965 a and 965 b are similar to LED light source 900 described inApplicant's copending U.S. patent application Ser. No. 10/627,386, thespecification of which has been incorporated herein by reference.

As explained above, computer 960 is programmed to control linearactuators 956 a, 957 a and 958 a of camera positioning system 955 a toprecisely position camera 959 a over any well in micro-well plate 901 a4. Likewise, computer 960 is programmed to control linear actuators 956b, 957 b and 958 b of camera positioning system 955 b to preciselyposition camera 959 b over any well in micro-well plate 901 a 1.Consequently, by making adjustments to the positions of the linearactuators, computer 960 positions camera 959 a over each well onmicro-well plate 901 a 4 and also positions camera 959 b over each wellof micro-well plate 901 a 1. In the preferred embodiment computer 960 isprogrammed to control the motion of camera positioning systems 955 a and955 b so that they can be positioned independently of each other. Afterthe camera has recorded an image of a well the data is transmitted backto computer 960 for analysis and data storage in a fashion similar tothat for the above described proteomic crystal verification andinspection system (see FIGS. 1-40).

Also, in the preferred embodiment, camera zoom lens 970 a is controlledby lens motor 971 a and filter wheel 972 a is controlled by wheel motor973 a. Likewise, camera zoom lens 970 b is controlled by lens motor 971b and filter wheel 972 b is controlled by wheel motor 973 b (FIGS. 85 aand 86).

In FIG. 87 drive wheel 975 (not shown) has turned and belt 962 has movedbar 963 a further to the right. Bar 963 a has pushed tray 901 a so thatmicro-well plate 901 a 2 is between camera 959 b and LED light source965 b and micro-well plate 901 a 5 is between camera 959 a and LED lightsource 965 a. Computer 960 controls the positions of camera 959 a andcamera 959 b so that each well of micro-well plates 901 a 2 and 901 a 5containing solution is observed and inspected. Data is transferred tocomputer 960 for analysis and storage.

In FIG. 88 drive wheel 975 (not shown) has turned and belt 962 has movedbar 963 a further to the right. Bar 963 a has pushed tray 901 a so thatmicro-well plate 901 a 3 is between camera 959 b and LED light source965 b and micro-well plate 901 a 6 is between camera 959 a and LED lightsource 965 a. Computer 960 controls the positions of camera 959 a andcamera 959 b so that each well of micro-well plates 901 a 3 and 901 a 6containing solution is observed and inspected. Data is transferred tocomputer 960 for analysis and storage.

In FIG. 89 micro-well plates 901 a 1-901 a 6 have all been inspected.Drive wheel 975 (not shown) has turned so that bar 963 b is pushedagainst tray 901 a.

In FIG. 90 bar 936 b has pushed tray 901 a to the rightmost position ofindexer 952 so that it is ready for removal by storage gantry 903.

Standby Station

In the preferred embodiment shown in FIGS. 83 and FIG. 85 a, proteomiccrystal verification and inspection system 951 includes standby station980. Standby station 980 is utilized as a positioning platform for thenext tray that is in line to be inspected by proteomic crystalverification and inspection system 951.

For example, FIG. 91 shows tray 901 b positioned on standby station 980after storage gantry 903 (FIG. 83) has placed it there. Micro-wellplates 901 a 2 and 901 a 5 on tray 901 a are being inspected by cameras959 b and 959 a, respectively.

In FIG. 92, storage gantry 903 (FIG. 83) has placed tray 901 a on thelower rail of standby station 980.

In FIG. 93, storage gantry 903 has returned tray 901 a to storage rack932 and storage gantry 903 has placed tray 901 b onto indexer 952 forinspection.

In FIG. 94, storage gantry 903 has placed tray 901 c onto the upper railof standby station 980. Tray 901 b is still being inspected by proteomiccrystal verification and inspection system 951. After tray 901 b isfinished being inspected, storage gantry 903 will place it on the lowerrail of standby station 980.

As demonstrated above, the utilization of standby station 980 decreasesthe amount of time that indexer 952 is empty thereby increasing theefficiency of the system.

Efficient System

In contrast to work cell area 904 (FIG. 59) of the earlier preferredembodiment, work cell area 950 does not require the utilization ofoverhead gantry 954 for the handling of the micro-well plates. This isbecause proteomic crystal verification and inspection system 951inspects the micro-well plates while they remain inside the micro-wellplate tray. As a result, there is a tremendous savings in time and costfor a much more efficient system. Also, there is lower likelihood thatmicro-well plates will become askew or damaged due to the fact that theydo not have to be separately handled by an overhead gantry system. Also,the embodiment shown in FIG. 83 has four cameras simultaneouslyinspecting the micro-well plates compared to the two cameras shown inFIG. 58.

Preferred Micro-well plate Holding Tray

Although a variety of designs for tray 901 a will work in conjunctionwith the present invention, a preferred micro-well plate holding tray901 a can be seen by reference to FIGS. 78-82. FIG. 78 shows aperspective view of a preferred micro-well plate holding tray 901 a witha left-rail 906 a, a top-rail 906 b, a right-rail 906 c, and abottom-rail 906 d. In the preferred embodiment the rectangulardimensions of tray 901 a are approximately 58 mm by 148 mm. The railsare preferably designed to be at or below the height of a typicalmicro-well plate. In the preferred embodiment, they are approximately 14mm in height. Also, in the preferred embodiment, tray 901 a can storesix micro-well plates. The storage tray is molded in one-piece ofthermoplastic resin and can be clear or colored. Color coding of thetrays adds visual feedback for sample identification. Six rectangularpockets 907 a-907 f (each one designed to hold a standard micro-wellplate) are formed in tray 901 a, between rails 906 a-906 d and interiorrails 908 a-908 d. Cut-down area 909 centered in top-rail 906 b providesaccess for a robotic gripper. For example, a robotic gripper can grasptop rail 906 b at cut-down area 909 and pull it or push it from or intoone of the storage slots shown in automated storage and retrieval system900 (FIGS. 41A and 41B). Thermoplastic tray 901 a is smooth on itsbottom side and can therefore efficiently slide over supporting surfacesso that the samples contained within the micro-well plate wells are notsubject to adverse jarring or adverse acceleration. Top-left-side cornerflat 910 provides orientation reference for tray 901 a. Roundedtop-right-side corner 912 is similar to the other two corners onbottom-rail 906 d of tray 901 a. Four second-cut-down areas 911 a-911 dprovide robotic access to allow for unstacking and stacking of thetrays.

FIG. 79 is an enlarged partial perspective front-side view of tray 901 ashowing pocket 907 a formed between top rail 906 b, interior rail 908 a,left rail 906 a and right rail 906 c and having bottom support 913. Thepreferred internal dimensions of the pockets between the enclosing railsare approximately 90 mm by 142 mm. Planar bottom support 913 hasrectangular recess forming a bottom-plate-shelf 914. Opening 915provides for structural flatness of the 901 a in the molding process.Four robotic access cut-outs 916 a-916 d, allow a robotic gripper toextend the horizontal plane of bottom-plate-shelf 914 and grasp amicro-well plate within pocket 907 a. For example, robotic grippers onwork cell gantry 905 can grab a micro-well plate in tray 901 a and placeit on either of the two proteomic crystal verification and inspectionsystems 100, shown in FIG. 43. Eight tapered guide pillars 917 a-917 hprovide for centering and guiding a micro-well plate into 901 a when themicro-well plate is placed into the tray. The tapered guide pillars 917a-917 h have their side facing the interior of the pocket sloped atpreferably 5 degrees away from normal to the planar bottom-support 913such that as the micro-well plate is placed within the pocket, thetapers help to center the plate as it is set onto the recessedbottom-plate shelf 914. The guiding features of pillars 917 a-917 h worktogether with the access cutouts 916 a-916 d to allow easy roboticstoring and retrieval of the plates, even though micro-well plates mayhave slightly different dimensions from manufacturer to manufacture.

FIG. 80 shows an enlarged cross-sectional view of section A-A of FIG. 79showing the planar bottom support 913 with its rectangular recessforming a bottom-plate-shelf 914. A second shelf 918 b is formed in therim of the rails of the tray such that groove 918 a formed in theback-side of tray 901 a will locate and properly nest into the secondshelf 918 b of a second tray when two or more trays are stackedtogether, one on top of the other.

FIG. 81 shows a bottom view of tray 901 a showing the six open areas,corner flat 910, rounded corner 912, robotic access cut-outs 916 a-916x, and groove 918 a for nesting of trays.

FIG. 82 is an enlarged partial perspective view of tray 901 a holding a96 well micro-well plate 919 and positioned to illustrate the visibilityof bar code 220 for a bar-code identifier for the plate. A properlypositioned external bar-code reader may read the barcode of themicro-well plate while the plate is contained within the pocket in thetray. The cut-down area 909 in the top-rail of the tray provides accessfor a robotic tray gripper. The top-left-side corner flat 910 providesorientation reference for the tray. The rounded top-right-side corner912 is similar to the other two corners on a bottom side of the tray.

Although the above-preferred embodiments have been described withspecificity, persons skilled in this art will recognize that manychanges to the specific embodiments disclosed above could be madewithout departing from the spirit of the invention. For example,although in the above preferred embodiments automated storage andretrieval system 900 was shown as being used in conjunction with theautomated receiving machine identified as proteomic crystal verificationand inspection system 100, automated storage and retrieval system 900could be used to store and retrieve a variety of subject matter for avariety of purposes other than proteomic crystal verification andinspection. For example, in another preferred embodiment, the twoprotein crystal verification and inspection systems 100 could bereplaced with two other automated receiving machines. In one preferredembodiment, the two protein crystal verification and inspection systems100 are replaced with two automated micro-well plate filling machinessimilar to the one disclosed in Applicant's U.S. patent application Ser.No. 09/702,164. In this preferred embodiment, storage gantry 903 removestrays containing empty micro-well plates from storage rack 932 anddelivers the trays to work cell gantry 905. Work cell gantry 905sequentially transfers the trays to the automated micro-well platefilling machines in a fashion similar to that described above inreference to the transfer of trays to the protein crystal verificationand inspection systems 100. After the micro-well plates have beenfilled, storage gantry 903 delivers the trays to storage rack 932 forstorage. Also, although access drawer 902 was discussed in detail in theabove preferred embodiments, it would also be possible to utilize othertypes of access devices other than the access drawer described fordepositing and removing trays. For example, the above invention could beconfigured so that an operator can insert trays into automated storageand retrieval system 900 by placing them on a flat table. Storage gantry903 would then remove them from the flat table and store the trays instorage rack 932. Also, although the above preferred embodimentsdiscussed the utilization of two protein crystal verification andinspection systems 100, it would also be possible to utilize just one ofthe two protein crystal verification and inspection system 100. Also,although the above preferred embodiments showed preferred tray 901 abeing utilized with automated storage and retrieval system 900, it couldalso be used with a variety of other automated machines that haverobotic grippers. Also, although the above preferred embodimentsdiscussed work cell gantry 905 moving micro-well plates, work cellgantry 905 could also move other types of subject matter stored in traysother than micro-well plates. For example, work cell gantry 905 couldalso move pharmaceuticals, disposable health care products such ascatheters and syringes, and biological compounds in micro-well plates orin individual bottles in storage trays, or the tray itself could be anindividual micro-well plate handled directly by the gantry. In addition,though the present invention is described in terms of 6 micro-wellplates to a tray, any number of micro-well plates could be handled withthe appropriately sized tray, from one to 24 or more. Also, although theabove preferred embodiments showed a plurality of micro-well platesstored on trays, it would be possible for the storage gantry, overheadgantry and proteomic crystal verification and inspection system tosequentially handle and inspect individual micro-well plates one at atime that are not positioned on a tray. Also, although the embodimentshown in FIGS. 84 shows a single computer 960 controlling camerainspection systems 955 a and 955 b, it would also be possible for eachcamera inspection system to have its own dedicated computer for control.Also, with regards to the proteomic crystal verification and inspectionsystem, although the above preferred embodiments specifically describean indexing device in which linear actuators 115, 150, and 160 operatein conjunction to sequentially position protein crystals under cameras155 and 135, there are a variety of other types of robotic indexingdevices that could also be utilized for the same purpose. For example,an indexing device could be built in which the plurality of micro-wellplates are kept in a stationary position. The camera lens would beattached to an indexing device that is preferably capable ofunrestricted movement in the horizontal plane. The camera lens would bemoved sequentially from micro-well to micro-well in the horizontalplane. Once in position over a micro-well, the lens could be raised orlowered in the vertical direction to achieve proper zoom and focus. Inanother embodiment, an indexing device could be built in which cameras155 and 135 are kept stationary with respect to horizontal movement. Inthis embodiment, the plurality of micro-well plates would be preferablyplaced on a positioning platform that is capable of unrestrictedmovement in the horizontal plane. In this fashion, the positioningplatform could be moved so that each micro-well is sequentiallypositioned underneath the appropriate camera. As with the previousembodiment, once in position over a micro-well, the lens could be raisedor lowered in the vertical direction to achieve proper zoom and focus.Also, although the first preferred embodiment discussed inspectingcrystals, grown by the hanging drop method, other crystals grownutilizing other methods could be inspected with equal effectiveness. Forexample, FIG. 12 shows protein crystal growth as a result of aqueousdrop in oil protein crystallization. Cameras 135 and 155 focus on thecrystals in drop 362. Also, although the preferred embodiment shown inAlso, although the above preferred embodiments discussed in detail howthe present invention is utilized for inspecting protein crystals insidedrops of liquid, the present invention could also be utilized to inspectother types of microscopic specimens. For example, the present inventioncould be utilized to inspect typical micro-well micro titer platereactions wherein the quality of the reaction can be judged by theamount and wavelength of fluorescence emitted by the specimen byconfiguring the system with appropriate light sources, filters, andsensitive cameras as is typical for fluorescence detection. Also,although the above preferred embodiments disclosed the utilization oftwo cameras 135 and 155, it would also be possible to have just onecamera that is capable of zooming out so that it can focus on the entirewell and zooming in so that it can focus on the drop of liquidcontaining the crystal. In addition, although an area CCD camera isshown, a linear CCD camera combined with moving of the micro-well platewould also work in the present invention. Also, in another preferredembodiment the detents 510 and 520 can be simply spring loaded and notcontrolled by the computer 105. Also, the invention is taught with alight panel light source, and or a bulb, likewise LED light sources andlaser light sources could also be used with the present invention.Although the system is shown that only moves the micro-well plates inone axis and the camera in the other two axes, the invention couldlikewise be practiced with either the micro-wells moving in twoorthogonal axes (such as X and Y) while the camera moves only in theZ-axis or the motion of all three axes be done with the camera system,wherein the micro-well plates are stationary and the system moves abovethem. These other variations of system design could also requirerearrangement of the light source or multiple light sources. Also, otherfilter types may be substituted for second filter 354. For example, alinearly polarized filter would be very effective. Also, although theabove preferred embodiments disclosed specific types of cameras 135 and155, other CCD cameras may be used in the present invention with lessresolution or with greater resolution and still practice the presentinvention. For example, cameras of 2,000 by 2,000 pixels and even 4,000by 4000 pixels are commercially available from several vendors. Whendigitizing these alternative cameras, the digitized image would have thecorresponding resolution of the camera. Also, one may practice thisinvention and digitize to greater gray-scale accuracy than 8-bit andgain advantage if the camera supports the greater bit depth, for exampleif the camera were cooled to reduce image noise. Therefore, the attachedclaims and their legal equivalents should determine the scope of theinvention.

1. An automated storage and retrieval device for trays holding subjectmatter, comprising: A) an access device for the insertion and removal ofa plurality of trays, B) a storage rack comprising a plurality ofvertically aligned storage slots for vertically storing a plurality oftrays, C) at least one automated machine, D) a storage gantry forvertical and horizontal movement of said plurality of trays between saidstorage rack, said access device and said at least one automatedmachine, said storage gantry being adopted to remove trays one-at-a-timefrom any one of said plurality of vertically aligned storage slots,transport the removed tray to said at least one automated machine,remove trays one-at-a-time from said at least one automated machine, andto return the removed tray to any one of said plurality of verticallyaligned storage slots, and D) at least one computer system programmed tocontrol said storage gantry.
 2. The automated storage and retrievaldevice as in claim 1, wherein said at least one automated machine is aninspection device.
 3. The automated storage and retrieval device as inclaim 2, wherein said inspection device is a device for inspecting andclassifying a plurality of microscopic crystals.
 4. The automatedstorage and retrieval device as in claim 3, wherein said at least oneinspection device comprises: E) at least one camera, F) an indexingdevice for sequentially placing said microscopic crystals in camera-viewof said at least one camera, and G) at least one control computerprogrammed to control said indexing device and said at least one camera,wherein said at least one control computer is programmed to receive fromsaid at least one camera images of said plurality of microscopiccrystals.
 5. The automated storage and retrieval device as in claim 4,wherein said at least one control computer automatically classifies saidplurality of microscopic crystals after receiving said images.
 6. Theautomated storage and retrieval device as in claim 1, wherein saidautomated machine comprises: E) at least one camera, F) an indexingdevice for receiving said plurality of trays and for placing saidsubject matter in camera view of said at least one camera, G) at leastone control computer programmed to control said indexing device and saidat least one camera, wherein said at least one control computer isprogrammed to receive from said at least one camera images of saidsubject matter.
 7. The automated storage and retrieval device as inclaim 6, wherein said automated machine further comprises an LED lightsource for illuminating said subject matter.
 8. The automated storageand retrieval device as in claim 1, wherein said at least one automatedmachine is an automated micro-well plate filling machine.
 9. Theautomated storage and retrieval device as in claim 1, wherein said atleast one automated machine comprises: E) a micro-well plate fillingassembly, comprising:
 1. an indexing device, and
 2. a fill mechanism incommunication with a media source and positioned to insert portions ofsaid media into the empty micro-well plates, and F) an automatic controlunit programmed to cause said indexing device to move empty micro-wellplates adjacent to said fill mechanism, and to cause said fill mechanismto inject media from said media source into wells in the micro-wellplates.
 10. The automated storage and retrieval device as in claim 1,wherein said subject matter is solution inside at least one micro-wellplate.
 11. The automated storage and retrieval device as in claim 10,wherein said at least one micro-well plate comprises a bar code, whereinsaid automated storage and retrieval device further comprises at leastone bar code reader in communication with said at least one computersystem.
 12. The automated storage and retrieval device as in claim 1,wherein said plurality of trays holds at least one micro-well plate,wherein said storage gantry comprises at least one robotic gripper,wherein said plurality of trays comprises: E) at least one cut-downaccess area for said at least one robotic gripper, F) a corner flat fortray orientation, and G) a plurality of tapered guide pillars forguiding said at least one micro-well plate into said plurality of trayswhen an automated robotic device places said at least one micro-wellplate into said plurality of trays.
 13. A method for automaticallystoring and retrieving a plurality of trays, wherein said plurality oftrays holds subject matter, comprising the steps of: A) placing aplurality of trays into an access device for the insertion and removalof a plurality of trays, B) transferring said plurality of trays fromsaid access device to a storage rack, comprising a plurality ofvertically aligned storage slots, C) transferring via a storage gantryutilizing vertical and horizontal movement said plurality of trays fromsaid storage rack to an automated machine, said storage gantry beingadopted to remove trays one-at-a-time from any one of said plurality ofvertically aligned storage slots, transport the removed tray to said atleast one automated machine, remove trays one-at-a-time from said atleast one automated machine, and to return the removed tray to any oneof said plurality of vertically aligned storage slots, and D)transferring via said storage gantry utilizing vertical and horizontalmovement said plurality of trays from said automated machine to saidstorage rack, and E) controlling the movement of said storage gantry viaat least one programmed computer system.
 14. The method as in claim 13,wherein said subject matter is a plurality of microscopic crystals andat least one automated machine is a device for inspecting andclassifying said plurality of microscopic crystals.
 15. The method as inclaim 14, wherein said at least one automated machine comprises: E) atleast one camera, F) an indexing device for sequentially placing saidmicroscopic crystals in camera-view of said at least one camera, and G)at least one control computer programmed to control said indexing deviceand said at least one camera, wherein said at least one control computeris programmed to receive from said at least one camera images of saidplurality of microscopic crystals, wherein said at least one controlcomputer is programmed to classify said plurality of microscopiccrystals.
 16. The method as in claim 15, wherein said at least onecontrol computer automatically classifies said plurality of microscopiccrystals after receiving said images.
 17. The method as in claim 13,wherein said at least one automated machine is an automated micro-wellplate filling machine.
 18. The method as in claim 13, wherein said atleast one automated machine comprises: E) a micro-well plate fillingassembly, comprising:
 1. an indexing device, and
 2. a fill mechanism incommunication with a media source and positioned to insert portions ofsaid media into the empty micro-well plates, and F) an automatic controlunit programmed to cause said indexing device to move empty micro-wellplates adjacent to said fill mechanism, and to cause said fill mechanismto inject media from said media source into wells in the micro-wellplates.
 19. The method as in claim 13, wherein said plurality of traysholds at least one micro-well plate.
 20. The method as in claim 19,wherein said at least one micro-well plate comprises a bar code, whereinsaid automated storage and retrieval device further comprises at leastone bar code reader in communication with said at least one computersystem.
 21. The method as in claim 13, wherein said plurality of traysholds at least one micro-well plate, wherein said storage gantrycomprises at least one robotic gripper, wherein said plurality of trayscomprises: E) at least one cut-down access area for said at least onerobotic gripper, F) a corner flat for tray orientation, and G) aplurality of tapered guide pillars for guiding said at least onemicro-well plate into said plurality of trays when an automated roboticdevice places said at least one micro-well plate into said plurality oftrays.
 22. The method as in claim 13, wherein said automated machinecomprises: E) at least one camera, F) an indexing device for receivingsaid plurality of trays and for placing said subject matter in cameraview of said at least one camera, G) at least one control computerprogrammed to control said indexing device and said at least one camera,wherein said at least one control computer is programmed to receive fromsaid at least one camera images of said subject matter.
 23. An automatedstorage and retrieval device for trays holding subject matter,comprising: A) an access device means for the insertion and removal of aplurality of trays, B) a storage rack means comprising a plurality ofvertically aligned storage slots for vertically storing a plurality oftrays, C) at least one automated machine means, D) a storage gantrymeans for vertical and horizontal movement of said plurality of traysbetween said storage rack means, said access device means and said atleast one automated machine means, said storage gantry means beingadopted to remove trays one-at-a-time from any one of said plurality ofvertically aligned storage slots, transport the removed tray to said atleast one automated machine, remove trays one-at-a-time from said atleast one automated machine, and to return the removed tray to any oneof said plurality of vertically aligned storage slots, and E) at leastone computer system means programmed to control said storage gantrymeans.
 24. The automated storage and retrieval device as in claim 23,wherein said at least one automated machine means is an inspectionmeans.
 25. The automated storage and retrieval device as in claim 24,wherein said inspection means is a device for inspecting and classifyinga plurality of microscopic crystals.
 26. The automated storage andretrieval device as in claim 25, wherein said at least one inspectionmeans comprises: E) at least one camera, F) an indexing means forsequentially placing said microscopic crystals in camera-view of said atleast one camera, and G) at least one control computer programmed tocontrol said indexing means and said at least one camera, wherein saidat least one control computer is programmed to receive from said atleast one camera images of said plurality of microscopic crystals. 27.The automated storage and retrieval device as in claim 26, wherein saidat least one control computer automatically classifies said plurality ofmicroscopic crystals after receiving said images.
 28. The automatedstorage and retrieval device as in claim 23, wherein said automatedmachine means comprises: E) at least one camera, F) an indexing meansfor receiving said plurality of trays and for placing said subjectmatter in camera view of said at least one camera, G) at least onecontrol computer programmed to control said indexing means and said atleast one camera, wherein said at least one control computer isprogrammed to receive from said at least one camera images of saidsubject matter.
 29. The automated storage and retrieval device as inclaim 28, wherein said automated machine means further comprises an LEDlight source for illuminating said subject matter.
 30. The automatedstorage and retrieval device as in claim 23, wherein said at least oneautomated machine means is an automated micro-well plate fillingmachine.
 31. The automated storage and retrieval device as in claim 23,wherein said at least one automated machine means comprises: E) amicro-well plate filling assembly, comprising:
 1. an indexing device,and
 2. a fill mechanism in communication with a media source andpositioned to insert portions of said media into the empty micro-wellplates, and F) an automatic control unit programmed to cause saidindexing device to move empty micro-well plates adjacent to said fillmechanism, and to cause said fill mechanism to inject media from saidmedia source into wells in the micro-well plates.
 32. The automatedstorage and retrieval device as in claim 23, wherein said subject matteris solution inside at least one micro-well plate.
 33. The automatedstorage and retrieval device as in claim 32, wherein said at least onemicro-well plate comprises a bar code, wherein said automated storageand retrieval device further comprises at least one bar code reader incommunication with said at least one computer system.
 34. The automatedstorage and retrieval device as in claim 23, wherein said plurality oftrays holds at least one micro-well plate, wherein said storage gantrycomprises at least one robotic gripper, wherein said plurality of trayscomprises: E) at least one cut-down access area for said at least onerobotic gripper, F) a corner flat for tray orientation, and G) aplurality of tapered guide pillars for guiding said at least onemicro-well plate into said plurality of trays when an automated roboticdevice places said at least one micro-well plate into said plurality oftrays.
 35. A method for growing protein crystals, comprising the stepsof: A) inserting drops of protein solution into a plurality ofmicro-well plates, B) placing said plurality of micro-well plates onto aplurality of Ways, C) placing said plurality of trays into an accessdevice for the insertion and removal of a plurality of trays, C)transferring said plurality of trays from said access device to astorage rack comprising a plurality of vertically aligned storage slots,D) transferring via a storage gantry utilizing vertical and horizontalmovement said plurality of Ways from said storage rack to an automatedmachine, said storage gantry being adopted to remove trays one-at-a-timefrom any one of said plurality of vertically aligned storage slots,transport the removed tray to said at least one automated machine,remove trays one-at-a-time from said at least one automated machine, andto return the removed tray to any one of said plurality of verticallyaligned storage slots, E) analyzing via said automated machine saiddrops of protein solution, F) transferring via said storage gantryutilizing vertical and horizontal movement said plurality of trays fromsaid automated machine to said storage rack, and G) controlling themovement of said storage gantry via at least one programmed computersystem.
 36. Protein crystals grown by a method comprising the steps of:A) inserting drops of protein solution into a plurality of micro-wellplates, B) placing said plurality of micro-well plates onto a pluralityof trays, C) placing said plurality of trays into an access device forthe insertion and removal of a plurality of trays, D) transferring saidplurality of trays from said access device to a storage rack, comprisinga plurality of vertically aligned storage slots, D) transferring via astorage gantry utilizing vertical and horizontal movement said pluralityof trays from said storage rack to an automated machine, said storagegantry being adopted to remove trays one-at-a-time from any one of saidplurality of vertically aligned storage slots, transport the removedtray to said at least one automated machine, remove trays one-at-a-timefrom said at least one automated machine, and to return the removed trayto any one of said plurality of vertically aligned storage slots, E)analyzing via said automated machine said drops of protein solution, F)transferring via said storage gantry utilizing vertical and horizontalmovement said plurality of trays from said automated machine to saidstorage rack, and G) controlling the movement of said storage gantry viaat least one programmed computer system.
 37. An automated storage andretrieval device for at least one micro-well plate holding subjectmatter, comprising: A) an access device for the insertion and removal ofa plurality of trays, B) a storage rack comprising a plurality ofvertically aligned storage slots for vertically storing said at leastone micro-well plate, C) at least one automated machine, D) a storagegantry for vertical and horizontal movement of said at least onemicro-well plate between said storage rack, said access device and saidat least one automated machine, said storage gantry being adopted toremove trays one-at-a-time from any one of said plurality of verticallyaligned storage slots, transport the removed tray to said at least oneautomated machine, remove trays one-at-a-time from said at least oneautomated machine, and to return the removed tray to any one of saidplurality of vertically aligned storage slots, and D) at least onecomputer system programmed to control said storage gantry.
 38. Theautomated storage and retrieval device as in claim 37, wherein said atleast one automated machine is an inspection device comprising: E) atleast one camera, F) an indexing device for sequentially placing said atleast one micro-well plate m camera-view of said at least one camera,and G) at least one control computer programmed to control said indexingdevice and said at least one camera, wherein said at least one controlcomputer is programmed to receive from said at least one camera imagesof said subject matter in said at least one micro-well plate.
 39. Theautomated storage and retrieval device as in claim 1, wherein saidstorage gantry is adapted to remove a tray from any one of saidplurality of vertically aligned storage slots and transport the tray tosaid automated machine without disturbing any other tray.
 40. Theautomated storage and retrieval device as in claim 1, wherein saidstorage gantry is adapted to transport a tray from said automatedmachine to any one of said plurality of storage slots that are empty oftrays without disturbing any other tray.
 41. The automated storage andretrieval device as in claim 1, wherein said storage gantry isconfigured to transport a tray directly to said at least one automatedmachine without first transporting the tray to a conveyor belt.
 42. Theautomated storage and retrieval device as in claim 1, wherein said atleast one computer system is programmed to: E) record the location ofeach tray of said plurality of trays within said storage rack, F)automatically control said storage gantry to remove a specific tray froma specific location within said storage rack, G) automatically controlsaid storage gantry to transport said specific tray to said at least oneautomated machine, H) automatically control said storage gantry toremove said specific tray from said at least one automated machine, I)automatically control said storage gantry to return said specific trayto said specific location within said storage rack, wherein steps F-Iare automatically executed at preprogrammed time intervals for thepurpose of observing protein crystal growth and said preprogrammed timeintervals are of sufficient length so as to be able to observe proteincrystal growth.
 43. The automated storage and retrieval device as inclaim 1, wherein said at least one computer system is programmed toautomatically identify the subject matter in each tray after the trayhas been placed in said access device and prior to transfer to saidstorage gantry for storage in said storage rack, wherein said computersystem is programmed to record the location and the identity of thesubject matter after said storage gantry has placed said tray in saidstorage rack.
 44. The automated storage and retrieval device as in claim43, wherein said at least one computer system automatically identifiesthe subject matter by utilization of a bar code reader in communicationwith said at least one computer system.
 45. The automated storage andretrieval device as in claim 42, wherein said preprogrammed timeintervals are approximately two week intervals.
 46. The automatedstorage and retrieval device as in claim 1, further comprising a standbystation for positioning at least one of said plurality of trays whileother of said plurality of trays are being handled by said at least oneautomated machine or transported by said storage gantry, wherein saidstandby station increases the efficiency of said at least one automatedmachine by decreasing the amount of time that said at least oneautomated machine is empty.